Logstash jdbc将外连接作为子文档(Logstash jdbc left outer join as subdocuments)

我正在使用Logstash jdbc插件将MySQL数据导入ElasticSearch。 由于左外连接，我最终得到一个'父行'的多个'子行'。 假设1个用户有1个或更多文档。 我尝试group_concat文档的文本，然后按用户ID分组以保留每个用户1行。

但是，MySQL的group_concat的长度限制为1024 ...

有没有人知道完全克服group_concat并将左外连接作为嵌套文档处理的解决方案？

感谢名单

I'm using the Logstash jdbc plugin to get MySQL data into ElasticSearch. Due to a left outer join I end up with multiple 'child rows' for a single 'parent row'. Say 1 user has 1 or more documents. I tried to group_concat the text of the documents and then group by by the user id to retain 1 row per user.

However, MySQL's group_concat has a length limit of 1024...

Does anyone know a solution to overcome the group_concat altogether and deal with left outer joins as nested documents?

Thanx

最满意答案

默认值为1024，但您可以使用增加GROUP_CONCAT的允许长度

SET SESSION group_concat_max_len = 102400

或您认为必要的长度（ 官方文档中的更多信息）

我在复杂的OUTER JOIN中使用它来创建许多子对象/文档，到目前为止它工作得非常好。

The default value is 1024, but you may increase the allowed length for GROUP_CONCAT using

SET SESSION group_concat_max_len = 102400

or whatever length you deem necessary (more info in the official docs)

I'm using that in complex OUTER JOINs which create many child objects/documents, and it works pretty fine so far.

更多推荐