Install and configure Logstash #

Logstash is an open-source data collection and processing tool. It’s used to collect, parse, enrich, and transform log data from various sources, including log files, event streams, and databases.

Install Logstash #

Download Logstash from the official Elastic website and install it per the instructions.

Configure Logstash #

A standard Logstash configuration consists of three sections: input, filter, and output. The input and filter sections depend on the sources of logs.

To send logs to VNETWORK Managed Logging, configure Logstash with Kafka output and enable the Kafka Integration Plugin in your Logstash installation.

1. Configure Logstash with Kafka output by adding the following data to the logstash.conf file:

output { kafka { codec => json topic_id => "yourlogin.yourtopic" bootstrap_servers => "endpoint" sasl_mechanism => "SCRAM-SHA-512" security_protocol => "SASL_SSL" sasl_jaas_config => "org.apache.kafka.common.security.scram.ScramLoginModule required username='yourlogin'password='yourpassword'; " key_serializer => "org.apache.kafka.common.serialization.StringSerializer" value_serializer => "org.apache.kafka.common.serialization.StringSerializer" } }

2. Customize the highlighted values:

yourlogin.yourtopic: Your username and topic name, separated with a dot (.)
endpoint: Kafka endpoint
yourlogin.yourtopic: Your login
yourpassword: Your password

You can find your username, password, login, and topic name information in the VNETWORK Customer Portal on the Logging page. Learn more about logging configuration in our dedicated guide.

For more settings, check out the Kafka output plugin documentation.

topic_id: ID of the Kafka topic to which Logstash will publish messages
bootstrap_servers: A comma-separated list of Kafka broker addresses
sasl.mechanism: Authentication mechanism that helps to verify a username and password entered to sign in your logs storage
security_protocol: Security protocol used to encrypt your data and protect it from theft
sasl_jaas_config: JAAS configuration required for SASL authentication
key_serializer: Serialization format used to convert keys of messages sent to Kafka
value_serializer: Serialization method applied to the content of messages before they are transmitted to Kafka

3. Save the changes in the Logstash configuration file.

4. Restart Logstash. Logstash will start sending logs to VNETWORK Managed Logging.