2020년 9월 1일 화요일

Logstash HTTP API

잘 몰랐는데 로그스태시는 시스템 정보 등을 제공할 때 사용되는 HTTP API가 기본 활성화되어 있다.

logstash.yml

다음은 메인으로 사용중인 6.4 버전의 정보를 요청한 결과.
C:\Users\Administrator>curl -XGET localhost:9600?pretty
{
  "host" : "MHKANG",
  "version" : "6.4.0",
  "http_address" : "127.0.0.1:9600",
  "id" : "4c5d01b2-f018-4681-a55a-236893217cd9",
  "name" : "MHKANG",
  "build_date" : "2018-08-18T00:25:22Z",
  "build_sha" : "f8014ac54e6c8ff6c071c0960ca1b00e9735f43a",
  "build_snapshot" : false
}

다음은 7.9 버전 로그스태시 테스트하다 발생한 에러.
PS D:\ELK\logstash-7.9.0\bin> .\logstash -f D:\test.conf
Sending Logstash logs to D:/ELK/logstash-7.9.0/logs which is now configured via log4j2.properties
[2020-09-01T22:20:01,121][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.9.0", "jruby.version"=>"jruby 9.2.12.0 (2.5.7) 2020-07-01 db01a49ba6 Java HotSpot(TM) 64-Bit Server VM 25.121-b13 on 1.8.0_121-b13 +indy +jit [mswin32-x86_64]"}
[2020-09-01T22:20:01,543][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-09-01T22:20:04,094][INFO ][org.reflections.Reflections] Reflections took 31 ms to scan 1 urls, producing 22 keys and 45 values
[2020-09-01T22:20:05,850][WARN ][logstash.javapipeline    ][main] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2020-09-01T22:20:05,945][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["D:/test.conf"], :thread=>"#"}
[2020-09-01T22:20:06,618][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.67}
[2020-09-01T22:20:07,032][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2020-09-01T22:20:07,134][INFO ][filewatch.observingtail  ][main][b21fb5559736d5cedd6e15324c8d4c57316d4b2011555970c2884ac7685753d5] START, creating Discoverer, Watch with file and sincedb collections
[2020-09-01T22:20:07,140][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
warning: thread "Api Webserver" terminated with exception (report_on_exception is true):
SystemCallError: Unknown error (SystemCallError) - bind(2) for "127.0.0.1" port 9600
        initialize at org/jruby/ext/socket/RubyTCPServer.java:131
               new at org/jruby/RubyIO.java:876
  add_tcp_listener at D:/ELK/logstash-7.9.0/vendor/bundle/jruby/2.5.0/gems/puma-4.3.5-java/lib/puma/binder.rb:229
  add_tcp_listener at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/forwardable.rb:229
   start_webserver at D:/ELK/logstash-7.9.0/logstash-core/lib/logstash/webserver.rb:104
               run at D:/ELK/logstash-7.9.0/logstash-core/lib/logstash/webserver.rb:60
              each at org/jruby/RubyRange.java:526
   each_with_index at org/jruby/RubyEnumerable.java:1258
               run at D:/ELK/logstash-7.9.0/logstash-core/lib/logstash/webserver.rb:55
   start_webserver at D:/ELK/logstash-7.9.0/logstash-core/lib/logstash/agent.rb:416
[2020-09-01T22:20:07,827][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#, :backtrace=>["org/jruby/ext/socket/RubyTCPServer.java:131:in `initialize'", "org/jruby/RubyIO.java:876:in `new'", "D:/ELK/logstash-7.9.0/vendor/bundle/jruby/2.5.0/gems/puma-4.3.5-java/lib/puma/binder.rb:229:in `add_tcp_listener'", "D:/ELK/logstash-7.9.0/logstash-core/lib/logstash/webserver.rb:104:in `start_webserver'", "D:/ELK/logstash-7.9.0/logstash-core/lib/logstash/webserver.rb:60:in `block in run'", "org/jruby/RubyRange.java:526:in `each'", "org/jruby/RubyEnumerable.java:1258:in `each_with_index'", "D:/ELK/logstash-7.9.0/logstash-core/lib/logstash/webserver.rb:55:in `run'", "D:/ELK/logstash-7.9.0/logstash-core/lib/logstash/agent.rb:416:in `block in start_webserver'"]}
[2020-09-01T22:20:07,859][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit>

6.4 버전의 웹서버에 바인딩된 네트워크 정보와 충돌한 모양. (가끔 혼자서도 충돌 에러 발생) HTTP API를 disable(http.enabled: false)하거나 IP 또는 포트 정보를 바꿔주면 동시 사용이 가능하다.


다음은 7.9 버전의 HTTP API disable 후 실행 결과.
PS D:\ELK\logstash-7.9.0\bin> .\logstash -f D:\test.conf
Sending Logstash logs to D:/ELK/logstash-7.9.0/logs which is now configured via log4j2.properties
[2020-09-01T22:25:47,228][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.9.0", "jruby.version"=>"jruby 9.2.12.0 (2.5.7) 2020-07-01 db01a49ba6 Java HotSpot(TM) 64-Bit Server VM 25.121-b13 on 1.8.0_121-b13 +indy +jit [mswin32-x86_64]"}
[2020-09-01T22:25:47,440][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-09-01T22:25:49,255][INFO ][org.reflections.Reflections] Reflections took 47 ms to scan 1 urls, producing 22 keys and 45 values
[2020-09-01T22:25:50,992][WARN ][logstash.javapipeline    ][main] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2020-09-01T22:25:51,089][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["D:/test.conf"], :thread=>"#"}
[2020-09-01T22:25:51,768][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.67}
[2020-09-01T22:25:52,209][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2020-09-01T22:25:52,287][INFO ][filewatch.observingtail  ][main][b21fb5559736d5cedd6e15324c8d4c57316d4b2011555970c2884ac7685753d5] START, creating Discoverer, Watch with file and sincedb collections
[2020-09-01T22:25:52,295][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-09-01T22:25:52,340][INFO ][logstash.agent           ] HTTP API is disabled (`http.enabled=false`); webserver will not be started.

그런데 7.8 버전까지는 기본 설정 상태에서도 6.4 버전과 충돌한 적이 한번도 없었는데? 윈도우 문젠가?

관련 글

댓글 없음:

댓글 쓰기

크리에이티브 커먼즈 라이선스