Logstash 필터 date - 2nd

date 플러그인에서 시간 포맷 구문은 다음 두 가지.

H	hour of the day (24-hour clock) H minimal-digit hour. Example: 0 for midnight. HH two-digit hour, zero-padded if needed. Example: 00 for midnight.

대문자가 아닌 소문자 구문을 사용하면 어떻게 될까? 다음은 샘플.

01/Jan/2010:00:00:00
01/Jan/2010:01:00:00
01/Jan/2010:02:00:00
01/Jan/2010:03:00:00
01/Jan/2010:04:00:00
01/Jan/2010:05:00:00
01/Jan/2010:06:00:00
01/Jan/2010:07:00:00
01/Jan/2010:08:00:00
01/Jan/2010:09:00:00
01/Jan/2010:10:00:00
01/Jan/2010:11:00:00
01/Jan/2010:12:00:00
01/Jan/2010:13:00:00
01/Jan/2010:14:00:00
01/Jan/2010:15:00:00
01/Jan/2010:16:00:00
01/Jan/2010:17:00:00
01/Jan/2010:18:00:00
01/Jan/2010:19:00:00
01/Jan/2010:20:00:00
01/Jan/2010:21:00:00
01/Jan/2010:22:00:00
01/Jan/2010:23:00:00

다음은 파이프라인 설정.

input {
  file {
  path => "d:/test.log"
  start_position => "beginning"
  sincedb_path => "nul"
 }
}

filter {
 mutate { 
  strip => "message" 
  remove_field => [ "@version", "path", "host" ]
 }

 date { 
  match => [ "message", "dd/MMM/yyyy:hh:mm:ss" ]
  timezone => "UTC"
  remove_field => "message"
 }
}

output {
 stdout {}
}

실행 결과는 다음과 같다.

[2020-08-23T15:09:52,232][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
{
       "message" => "01/Jan/2010:00:00:00",
    "@timestamp" => 2020-08-23T06:09:52.228Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
    "@timestamp" => 2010-01-01T01:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T02:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T03:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T04:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T05:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T06:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T07:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T08:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T09:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T10:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T11:00:00.000Z
}
{
    "@timestamp" => 2010-01-01T00:00:00.000Z
}
{
       "message" => "01/Jan/2010:13:00:00",
    "@timestamp" => 2020-08-23T06:09:52.231Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
       "message" => "01/Jan/2010:14:00:00",
    "@timestamp" => 2020-08-23T06:09:52.232Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
       "message" => "01/Jan/2010:15:00:00",
    "@timestamp" => 2020-08-23T06:09:52.232Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
       "message" => "01/Jan/2010:16:00:00",
    "@timestamp" => 2020-08-23T06:09:52.232Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
       "message" => "01/Jan/2010:17:00:00",
    "@timestamp" => 2020-08-23T06:09:52.233Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
       "message" => "01/Jan/2010:18:00:00",
    "@timestamp" => 2020-08-23T06:09:52.233Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
       "message" => "01/Jan/2010:19:00:00",
    "@timestamp" => 2020-08-23T06:09:52.233Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
       "message" => "01/Jan/2010:20:00:00",
    "@timestamp" => 2020-08-23T06:09:52.233Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
       "message" => "01/Jan/2010:21:00:00",
    "@timestamp" => 2020-08-23T06:09:52.233Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
       "message" => "01/Jan/2010:22:00:00",
    "@timestamp" => 2020-08-23T06:09:52.233Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}
{
       "message" => "01/Jan/2010:23:00:00",
    "@timestamp" => 2020-08-23T06:09:52.233Z,
          "tags" => [
        [0] "_dateparsefailure"
    ]
}

01~12시를 제외한 나머지는 파싱 실패. 실패할거면 다 실패하든가 12시는 0시로 파싱하는 거 보니 12시간 단위 시간대로 파싱하는 듯. 샘플 다시 만들어야겠네.

관련 글

• Logstash 필터 date
• Logstash 필터 date - 3rd