Read The Docs has become the defacto standard for technical documentation, particularly in the Python world. You write your document content in reStructured Text (RST) markdown with a simple text editor then build online ready documentation…
While developing software, documentation is frequently the furthest thing from a programmer’s mind. Writing documentation can be tedious and, well let’s just say it, downright boring. But these days, open platforms supporting public APIs ar…
Apache NiFi is an open source data ingestion system that offers up a rich variety of components you can assemble to ingest, enrich, and store data. NiFi can grab data from any number of data sources, including logs and live streams. This ar…
Network packet capture and analysis are commonly done with tools like tcpdump, snort, and Wireshark. These tools provide the capability to capture packets live from networks and store the captures in PCAP files for later analysis. A much be…
Morphlines is an open source parsing engine that enables you to write parsing applications that can handle a wide variety of log formats. Part 1 of this series on log parsing with morphlines, I described how to use morphlines scripts to par…
Morphlines is an open source framework that eases the development of data ingestion and parsing applications written in Java. Originally developed by Cloudera, Morphlines is part of the Kite SDK which was spun off as its own open source pro…
It’s been awhile since Kibana 4 was released, so I figured it was about time I updated my OSSEC Log Management Console to use the latest and greatest Kibana. The look and feel of Kibana has changed quite a bit, with a new data discovery mod…
Although Eclipse is commonly used for Java web service programming, it can be challenging to configure and use for testing. Luckily Eclipse is not the only IDE game in town. NetBeans provides a rich environment for developing Java web servi…
Benjamin Franklin once wrote “…in this world nothing can be said to be certain, except death and taxes”. In this computerized world of ours, I would add having to backup your data to free up disk space to that list of eventualities. For Ela…
The ELK stack (Elasticsearch-Logstash-Kibana) provides a cost effective alternative to commercial SIEMs for ingesting and managing OSSEC alert logs. Previously I wrote a blog – OSSEC Log Management with Elasticsearch – that discusses the de…
The first article in this two part series focused on developing Elasticsearch clients with Perl. Elasticsearch also has an excellent Python library which lets you search for and analyze your data with one of the many mathematics and machine…
Since creating a log management system for the OSSEC HIDS with Elasticsearch, I have been busy applying this useful search technology in other projects. Elasticsearch is a marvelous system for ingesting streaming data that gets indexed on t…