Windowsのプロセス単位のCPU使用率アラートをfluentdを使ってやってみる…の続き
2014.08.27 追記
Release 0.10.49 で入っているTextFormatterを使って、plaintextformatterを使ってるところを無理やりjson固定でのTextFormatterに入れ替えることで、末尾にある文字コード変換のエラーは解消できるようになりました。
fluentd/ChangeLog at master · fluent/fluentd · GitHub
前回(Windowsのプロセス単位のCPU使用率アラートをfluentdを使ってやってみる - メモ帳みたいなもの)の続きです。
やってること自体はfluentdを使っている人であればありふれた内容なのでほぼ自分用のメモです。
nxlog.conf
im_msvistalogはExec でsyslog形式に変換して流しても動きましたが、情報が冗長という感じだったためim_fileと同じ形にしました。
fluent側で受けるtagを分けるために、Outputを追加。
どうも重複して利用がダメそうで、結局流れでInput/Processor/Output/Routeをすべて追加。
こんなんで良いのかな…。
define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension syslog> Module xm_syslog </Extension> <Extension json> Module xm_json </Extension> <Input in> Module im_file File "D:\work\winfluent\srclog\log.txt" SavePos TRUE InputType LineBased </Input> <Input ev> Module im_msvistalog # Exec $Message = to_json(); to_syslog_bsd(); SavePos TRUE ReadFromLast TRUE </Input> <Processor t> Module pm_transformer OutputFormat syslog_bsd Exec $Message=(": "+$raw_event); </Processor> <Processor t_ev> Module pm_transformer OutputFormat syslog_bsd Exec $Message=($raw_event); </Processor> <Output out> Module om_udp Host xxx.xxx.xxx.xxx Port 55514 </Output> <Output out_ev> Module om_udp Host xxx.xxx.xxx.xxx Port 55515 </Output> <Route r> Path in => t => out </Route> <Route r_ev> Path ev => t_ev => out_ev </Route>
追加プラグイン
/usr/lib64/fluent/ruby/bin/gem install fluent-plugin-numeric-monitor /usr/lib64/fluent/ruby/bin/gem install fluent-plugin-growthforecast
入れてるけどまだ使ってないプラグイン
/usr/lib64/fluent/ruby/bin/gem install fluent-plugin-file-alternative /usr/lib64/fluent/ruby/bin/gem install fluent-plugin-datacounter /usr/lib64/fluent/ruby/bin/gem install fluent-plugin-redeliver
td-agent.conf
im_fileとim_msvistalogでtagを変えたかったため、受けポートを増やして対応。
2回目のrewrite_tag_filterで、growthforecastに投げる際に「tag_for section」するとタグにPIDが入っているのが却って邪魔になったのでタグからPIDは外した。notifier のタグにPIDが入るのとメリット・デメリット差し引きでどちらかを取るか…という感じではある。
<source> type syslog protocol_type udp port 55514 tag winps </source> <source> type syslog protocol_type udp port 55515 tag winev </source> ################################################################## <match winev.**> type copy <store> type file path /var/log/td-agent/arch/winev time_slice_format %Y%m%d buffer_type file buffer_path /var/log/td-agent/buffer/winev/ buffer_chunk_limit 100m flush_interval 5s </store> </match> ################################################################## <match winps.**> type copy <store> type file path /var/log/td-agent/arch/winps buffer_type file buffer_path /var/log/td-agent/buffer/winps/ buffer_chunk_limit 100m flush_interval 5s </store> <store> type filter all deny allow message: /firefox/, message: /Idle/ </store> </match> <match filtered.**> type rewrite_tag_filter rewriterule1 host ^(.+)$ filterrewrited.$1.winps remove_tag_prefix filtered </match> <match filterrewrited.**> type parser remove_prefix filterrewrited add_prefix winproc format /^(?<Name>[^ ]* +\d+) +(?<Cpu>\d+) +(?<Thd>\d+) +(?<Hnd>\d+) +(?<Priv>\d+) +(?<CpuTime>.+) +(?<ElapsTime>.+)$/ key_name message suppress_parse_error_log true </match> <match winproc.**> type rewrite_tag_filter rewriterule1 Name ^([^ ]*) +(\d+)$ $1.${tag} remove_tag_prefix winproc </match> <match firefox.**> type copy <store> type notifier <def> pattern firefox check numeric_upward warn_threshold 5 crit_threshold 10 target_keys Cpu </def> </store> <store> type numeric_monitor count_interval 60 aggregate tag output_per_tag yes tag_prefix monitor monitor_key Cpu output_key_prefix cpu_stat percentiles 50 </store> </match> <match Idle.**> type copy <store> type notifier <def> pattern Idle check numeric_downward warn_threshold 95 crit_threshold 80 target_keys Cpu </def> </store> <store> type numeric_monitor count_interval 60 aggregate tag output_per_tag yes tag_prefix monitor monitor_key Cpu output_key_prefix cpu_stat percentiles 50 </store> </match> ################################################################## <match notification.**> type stdout # type mail # host localhost # port 25 # from FROM # to TO # subject fluentd notification # out_keys pattern,target_tag,target_key,level,value,message_time </match> #<match monitor.**> # type stdout #</match> <match monitor.**> type copy <store> type stdout </store> <store> type growthforecast remove_prefix monitor gfapi_url http://localhost:5125/api/ service cpustat tag_for section name_keys cpu_stat_max,cpu_stat_min,cpu_stat_avg,cpu_stat_percentile_50 </store> </match>
その他
fluent-plugin-file-alternative でイベントログをファイル保存しようとすると文字コード変換部でエラー。
ちょっと追いきれてないので、とりあえずメモだけ。out_file であれば問題なくファイルに保存できました。
2014-05-07 14:10:07 +0900 [warn]: emit transaction failed error_class=Encoding::UndefinedConversionError error=#<Encoding::UndefinedConversionError: "\xE3" from ASCII-8BIT to UTF-8> 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-mixin-plaintextformatter-0.2.6/lib/fluent/mixin/plaintextformatter.rb:85:in `encode' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-mixin-plaintextformatter-0.2.6/lib/fluent/mixin/plaintextformatter.rb:85:in `to_json' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-mixin-plaintextformatter-0.2.6/lib/fluent/mixin/plaintextformatter.rb:85:in `stringify_record' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-mixin-plaintextformatter-0.2.6/lib/fluent/mixin/plaintextformatter.rb:115:in `format' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/output.rb:527:in `block in emit' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/event.rb:54:in `call' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/event.rb:54:in `each' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/output.rb:518:in `emit' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/match.rb:36:in `emit' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/engine.rb:152:in `emit_stream' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/engine.rb:132:in `emit' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/plugin/in_syslog.rb:199:in `emit' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/plugin/in_syslog.rb:173:in `receive_data' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/plugin/in_syslog.rb:245:in `call' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/plugin/in_syslog.rb:245:in `on_read' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/cool.io-1.1.1/lib/cool.io/io.rb:108:in `on_readable' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/cool.io-1.1.1/lib/cool.io/io.rb:170:in `on_readable' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/cool.io-1.1.1/lib/cool.io/loop.rb:96:in `run_once' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/cool.io-1.1.1/lib/cool.io/loop.rb:96:in `run' 2014-05-07 14:10:07 +0900 [warn]: /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.45/lib/fluent/plugin/in_syslog.rb:118:in `run' 2014-05-07 14:10:07 +0900 [error]: syslog failed to emit error="\"\\xE3\" from ASCII-8BIT to UTF-8" error_class="Encoding::UndefinedConversionError" tag="winev.user.info" record="{\"host\":\"desktop-PC\",\"ident\":\"Service_Control_Manager\",\"pid\":\"672\",\"message\":\": 2014-05-06 20:59:12 nsr-PC INFO 7036 WWAN AutoConfig \u30B5\u30FC\u30D3\u30B9\u306F \u5B9F\u884C\u4E2D \u72B6\u614B\u306B\u79FB\u884C\u3057\u307E\u3057\u305F\u3002\\r\"}"