问题描述
由于此错误，我的一些CouchDB记录未通过管道传递到Elasticsearch：
错误：无效的字段引用：
[iso-8859-1]to
异常：Java :: OrgLogstash :: FieldReference :: IllegalSyntaxException
堆栈：org.logstash.FieldReference $ StrictTokenizer.tokenize（FieldReference.java:303）
这是什么意思，如何确定导致该错误的记录？
这是完整的错误日志：
logstash | Pipeline_id:main
logstash | Plugin: <LogStash::Inputs::CouchDBChanges ignore_attachments=>true,codec=><LogStash::Codecs::JSON id=>"json_17404f4f-4c86-4212-a2d5-caf57d2f057b",enable_metric=>true,charset=>"UTF-8">,password=><password>,port=>5984,host=>"host.domain",sequence_path=>"0",id=>"1ddcb1db7a5cbea8408c2c27145f7f295da0f1aaea7e510fc4a182a0b5dca7c1",db=>"incidents",username=>"admin",secure=>false,heartbeat=>1000,keep_id=>false,keep_revision=>false,always_reconnect=>true,reconnect_delay=>10>
logstash | Error: Invalid FieldReference: `[iso-8859-1]to`
logstash | Exception: Java::OrgLogstash::FieldReference::IllegalSyntaxException
logstash | Stack: org.logstash.FieldReference$StrictTokenizer.tokenize(FieldReference.java:303)
logstash | org.logstash.FieldReference.parse(FieldReference.java:204)
logstash | org.logstash.FieldReference.parsetoCache(FieldReference.java:195)
logstash | org.logstash.FieldReference.from(FieldReference.java:127)
logstash | org.logstash.FieldReference.lambda$from$0(FieldReference.java:118)
logstash | java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1737)
logstash | org.logstash.FieldReference.from(FieldReference.java:118)
logstash | org.logstash.ConvertedMap.convertKey(ConvertedMap.java:122)
logstash | org.logstash.ConvertedMap.access$000(ConvertedMap.java:44)
logstash | org.logstash.ConvertedMap$1.visit(ConvertedMap.java:55)
logstash | org.logstash.ConvertedMap$1.visit(ConvertedMap.java:49)
logstash | org.jruby.RubyHash.visitLimited(RubyHash.java:698)
logstash | org.jruby.RubyHash.visitAll(RubyHash.java:683)
logstash | org.logstash.ConvertedMap.newFromrubyHash(ConvertedMap.java:89)
logstash | org.logstash.ConvertedMap.newFromrubyHash(ConvertedMap.java:84)
logstash | org.logstash.Valuefier.lambda$initConverters$12(Valuefier.java:171)
logstash | org.logstash.Valuefier.convert(Valuefier.java:94)
logstash | org.logstash.ConvertedMap$1.visit(ConvertedMap.java:55)
logstash | org.logstash.ConvertedMap$1.visit(ConvertedMap.java:49)
logstash | org.jruby.RubyHash.visitLimited(RubyHash.java:698)
logstash | org.jruby.RubyHash.visitAll(RubyHash.java:683)
logstash | org.logstash.ConvertedMap.newFromrubyHash(ConvertedMap.java:89)
logstash | org.logstash.ConvertedMap.newFromrubyHash(ConvertedMap.java:84)
logstash | org.logstash.Valuefier.lambda$initConverters$12(Valuefier.java:171)
logstash | org.logstash.Valuefier.convert(Valuefier.java:94)
logstash | org.logstash.ConvertedMap$1.visit(ConvertedMap.java:55)
logstash | org.logstash.ConvertedMap$1.visit(ConvertedMap.java:49)
logstash | org.jruby.RubyHash.visitLimited(RubyHash.java:698)
logstash | org.jruby.RubyHash.visitAll(RubyHash.java:683)
logstash | org.logstash.ConvertedMap.newFromrubyHash(ConvertedMap.java:89)
logstash | org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_initialize(JrubyEventExtLibrary.java:92)
logstash | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_couchdb_changes_minus_3_dot_1_dot_6.lib.logstash.inputs.couchdb_changes.RUBY$method$build_event$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-couchdb_changes-3.1.6/lib/logstash/inputs/couchdb_changes.rb:235)
logstash | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_couchdb_changes_minus_3_dot_1_dot_6.lib.logstash.inputs.couchdb_changes.RUBY$block$run$4(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-couchdb_changes-3.1.6/lib/logstash/inputs/couchdb_changes.rb:176)
logstash | org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
logstash | org.jruby.runtime.BlockBody.yield(BlockBody.java:106)
logstash | org.jruby.runtime.Block.yield(Block.java:184)
logstash | org.jruby.RubyArray.each(RubyArray.java:1809)
logstash | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_couchdb_changes_minus_3_dot_1_dot_6.lib.logstash.inputs.couchdb_changes.RUBY$block$run$3(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-couchdb_changes-3.1.6/lib/logstash/inputs/couchdb_changes.rb:167)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.protocol.RUBY$method$call_block$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/protocol.rb:433)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.protocol.RUBY$method$\=\^\^$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/protocol.rb:424)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.protocol.RUBY$method$read$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/protocol.rb:129)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.response.RUBY$method$read_chunked$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http/response.rb:321)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.response.RUBY$block$read_body_0$1(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http/response.rb:285)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.response.RUBY$method$inflater$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http/response.rb:278)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.response.RUBY$method$read_body_0$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http/response.rb:283)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.response.RUBY$method$read_body$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http/response.rb:204)
logstash | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_couchdb_changes_minus_3_dot_1_dot_6.lib.logstash.inputs.couchdb_changes.RUBY$block$run$2(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-couchdb_changes-3.1.6/lib/logstash/inputs/couchdb_changes.rb:166)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.RUBY$block$transport_request$2(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http.rb:1513)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.response.RUBY$method$reading_body$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http/response.rb:165)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.RUBY$method$transport_request$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http.rb:1512)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.RUBY$method$request$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http.rb:1474)
logstash | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_couchdb_changes_minus_3_dot_1_dot_6.lib.logstash.inputs.couchdb_changes.RUBY$block$run$1(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-couchdb_changes-3.1.6/lib/logstash/inputs/couchdb_changes.rb:161)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.RUBY$method$start$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http.rb:914)
logstash | uri_3a_classloader_3a_.Meta_minus_INF.jruby_dot_home.lib.ruby.stdlib.net.http.RUBY$method$start$0(uri:classloader:/meta-inf/jruby.home/lib/ruby/stdlib/net/http.rb:609)
logstash | usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_couchdb_changes_minus_3_dot_1_dot_6.lib.logstash.inputs.couchdb_changes.RUBY$method$run$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-couchdb_changes-3.1.6/lib/logstash/inputs/couchdb_changes.rb:157)
logstash | usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:378)
logstash | usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0$__VaraRGS__(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)
logstash | org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
logstash | org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
logstash | org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)
logstash | usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_input$1(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:369)
logstash | org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)
logstash | org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
logstash | org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)
logstash | org.jruby.runtime.Block.call(Block.java:139)
logstash | org.jruby.RubyProc.call(RubyProc.java:318)
logstash | org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
logstash | java.base/java.lang.Thread.run(Thread.java:834)
logstash | [2020-09-14T19:09:15,305][INFO ][logstash.inputs.couchdbchanges][main][1ddcb1db7a5cbea8408c2c27145f7f295da0f1aaea7e510fc4a182a0b5dca7c1] Connecting to CouchDB _changes stream at: {:host=>"host.domain",:port=>"5984",:db=>"incidents"}
logstash | [2020-09-14T19:09:15,306][INFO ][logstash.inputs.couchdbchanges][main][1ddcb1db7a5cbea8408c2c27145f7f295da0f1aaea7e510fc4a182a0b5dca7c1] Using service uri : {:uri=>#<URI::HTTP http://host.domain:5984/incidents/_changes?Feed=continuous&include_docs=true&since=4586095-g1AAAAI7eJzLYWBg4MhgTmHgzcvPy09JdcjLz8gvLskBCjMlMiTJ____PyuDOYmBc3lGLlCM3cQy2SLJ3ARdPQ4TkhSAZJI93JA9P8GGmKZYGBmYEm2IA8iQeJghTCIVEJekJhkaGFsQa0gCyJB6uEvWrwMbkmpomJycSqxL8liAJEMDkAKaMx9i0GZ_sEGJKSZplmlGJBm0AGLQfohBu4zBBgG9lGpgnEySQQcgBt2HGLQ2DGKQYaqhWbIxSQY9gBgEDaPtAmCDzAxNzIwszdC1ZQEAodmvng&heartbeat=1000>}
logstash | [2020-09-14T19:09:15,319][ERROR][logstash.javapipeline ][main][1ddcb1db7a5cbea8408c2c27145f7f295da0f1aaea7e510fc4a182a0b5dca7c1] A plugin had an unrecoverable error. Will restart this plugin.
解决方法
如果仔细查看错误消息，它会告诉您在解析[iso-8859-1]to时出现了问题，logstash使用strictTokenizer来解析值，而this是准确的代码从哪里抛出此异常。
复制的代码来自同一链接，
// if we have encountered ambiguous syntax and are not in strict-mode,// fall back to legacy parser.
if (potentiallyAmbiguousSyntaxDetected) {
throw new FieldReference.IllegalSyntaxException(String.format("Invalid FieldReference: `%s`",reference.toString()));
}
正如您在注释中看到的那样，您的字符串[iso-8859-1]to的语法含糊不清，导致出现此异常，并且strictTokenizer的代码也存在于同一Java文件中，它是一个静态类，我快速浏览，但找不到关闭方法，因此可以避免此问题，但是如果要在strict mode中进行操作，另一种选择是在解析之前修复这些格式错误/含糊不清的字符串，也许可以grok filter。