During a review of log events coming into Elasticsearch I came across some that included ASCII colour codes in them. Below is one example where they are included in the log level, noting the  ‘‘box’’[39mDEBUG’‘box’’[0;39. While this is handy for colouring the log levels while viewing in a terminal, it is not so handy for use in Elasticsearch or Kibana. 

Some more examples, noting these were mutated to be all uppercase hence the capital ‘‘M’’.

To clean this up the following filter can be applied in the Logstash pipeline. In this case, I apply the gsub to both the message and log.level fields replacing the colour codes with nothing, thus removing them.

1
2
3
4
5
6
7

mutate {
  id => "mutate-ansii-colours"
  gsub => [
    "message", "\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|M|K]", "",
    "[log][level]", "\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|M|K]", ""
  ]
}

References

• Logstash mutate gsub documentation found here
• Some information about using ASCII colour codes to colour your logs by Wix Engineering here