Facing the growing threat of cybercrime, and to answer compliance requirements, more and more organizations are looking at their DevOps and Operations teams to implement security. The term “security”, however, often triggers negative feelings among engineers. The reason for this is that security is associated with siloed, sequential and complicated processes — all roadblocks to fast development and deployment.
We founded Logz.io four years ago with a vision to provide a platform for engineers like us to operate the next generation of applications and infrastructure. As engineers, we experienced firsthand the challenges of running large-scale, open-source tools and the frustration of dealing with slow, proprietary tools which were not designed to support the rapidly changing technology stack we operated.
While there are plenty of articles aiming to instruct newbies on “how to become a DevOps engineer”, none of them answer a simple question – should you become one to start with? For me, it’s been a long and winding journey to becoming what is called a Site Reliability Engineer (SRE). The journey was unplanned, and for each step, I had to figure out what I want my next challenge to be and how to achieve it.
In Elasticsearch parlance, a document is serialized JSON data. In a typical ELK setup, when you ship a log or metric, it is typically sent along to Logstash which groks, mutates, and otherwise handles the data, as defined by the Logstash configuration. The resulting JSON is indexed in Elasticsearch.
Most software and systems generate audit logs. They are a means to examine what activities have occurred on the system and are typically used for diagnostic performance and error correction. System Administrators, network engineers, developers, and help desk personnel all use this data to aid them in their jobs and maintain system stability. Audit logs have also taken on new importance for cybersecurity and are often the basis of forensic analysis, security analysis, and criminal prosecution.
In many projects, the product development workflow has three main concerns: building, testing, and deployment. Each change to the code means something could accidentally go wrong, so in order to prevent this from happening developers adopt many strategies to diminish incidents and bugs. Jenkins, and other continuous integration tools (CI) are used together with a source version software (such as GIT) to test and quickly evaluate the updated code.
Welcome to autumn! Here in New England we anxiously await the transition to winter as our leaves start to change color and the summer heat begins to temper. It’s been a busy few seasons for us – we’ve introduced new features and launched our new community efforts. Want to learn more? Read on!
Few things put more fear into the hearts of IT departments than the thought of a massive cyber attack — and for good reason. In 2017, the average cost of security breaches for U.S. organizations was estimated at $3.5 million. But it’s not all about the money. In addition to the financial burden caused by cyber attacks, the loss of client trust and credibility post-breach can be extremely difficult to bounce back from.
Franz Kafka was a German-speaking Bohemian Jewish novelist and short story writer, widely regarded as one of the major figures of 20th-century literature. Apache Kafka, on the other hand, is an open-source stream-processing software platform. Due to its widespread integration into enterprise-level infrastructures, monitoring Kafka performance at scale has become an increasingly important issue.
Last year we introduced Live Tail — the ability to see a live feed of all the logs in your system, in real time, within Kibana. This ability to see a live stream of logs as they are being outputted from the different processes in a monitored environment was a greatly requested feature, and since being introduced we have received some excellent feedback from users that has allowed us to improve the basic functionality of Live Tail.
