Nginx logs to Graylog, input extractors

To get the most out of Graylog we need to filter our log messages into separate fields. Those fields will later be used to create different searches which can be saved as a Dashboard widget.

Graylog Input extractors

Lets get started by going to System -> Inputs and press Manage extractors for our Nginx Access logs input. In the Manage Extractors page we just hit Get started to begin our journey to create our first extractor On the next page, just press Load Message to load the last log message received to that input. There already exists some fields in our log. But the part we are interested in is all contained in the message field. Click select extractor type on the message field and select Regular expression.

Our first extractor will just remove the first "crap" nginx logs, the result is a pretty JSON string we can use in our next extractor. The result from our regular expression filter is stored in the new field we call json. Use the Try button to make sure your regular expression works as intended. When you are done just hit Create extractor. Now we will get a new json field on our message. It does not work retroactive though, so you need to receive a new message for the field to appear.

When you have received a new message with our new json field. Go back to Manage Extractors and create a new one, after Load Message step click Select extractor type on the json field, and select JSON. Now you should have lots of fields in each of your messages. Makes it way easier to search and filter for stuff you want.