四时宝库

因为elk(Elasticsearch,logstash,Kibana是一家的),所以使用 logstash做为作为导入工具是最好的选择。

1))下载logstash https://www.elastic.co/cn/logstash

2)解压后进入logstash目录

3)新版本（2020/3/12，7.6.1）无需执行bin/logstash-plugin install logstash-input-jdbc,包内自带

4)新建导入的配置文件

vi cat config/mysql-to-elasticsearch.conf

input {
    
    jdbc {
      jdbc_connection_string => "jdbc:mysql://127.0.0.1:3786/datacapture?useUnicode=true&characterEncoding=UTF-8&serverTimezone=GMT%2B8&allowMultiQueries=true"
      jdbc_user => "root"
      jdbc_password => "123456"
      jdbc_driver_library => "/Users/hongyanma/repo/mysql/mysql-connector-java/5.1.46/mysql-connector-java-5.1.46.jar"
      jdbc_driver_class => "com.mysql.jdbc.Driver"
      jdbc_paging_enabled => "true"
      jdbc_page_size => "50000"
      jdbc_default_timezone => "Asia/Shanghai"
      statement_filepath => "/Users/hongyanma/gitspace/java/logstash-7.6.1/config/layer_video.sql"
      schedule => "* * * * *"
      type => "jdbc"
    }
}
 
 
output {
    elasticsearch {
        hosts => "127.0.0.1:9200"
       # protocol => "http"
        index => "datacapture"
        document_type => "video"
        document_id => "%{id}"
    }
    
}

vi /Users/hongyanma/gitspace/java/logstash-7.6.1/config/person_info.sql

select * from person_info

注意sql结尾不要加分号

4）运行logstash

bin/logstash -f config/mysql-to-elasticsearch.conf

启动需要一点时间，然后就会输出执行的sql