기록

Logstash 설치

• java 설치확인
• wget https://artifacts.elastic.co/downloads/logstash/logstash-7.10.1-amd64.deb
• sudo dpkg -i logstash-7.10.1-amd64.deb

logstash-simple.conf (간단한 테스트를 위한 파일작성)

•  실행
  • /usr/share/logstash/bin/logstash -f ~/logstash/logstash-simple.conf

fileBeat 설치(www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html)

• curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.10.1-amd64.deb
• sudo dpkg -i filebeat-7.10.1-amd64.deb
• /usr/share/filebeat/ 위치에 설치 확인

curator 설치

• apt install python-pip
• pip install elasticsearch-curator

curator-config.yml

delete.yml

•  명령어
  • /usr/local/bin/curator --config curator-config.yml --dry-run delete.yml
  • cron 등록 및 스케쥴링 job 생성

Kibana 설치(www.elastic.co/guide/en/kibana/7.10/deb.html#deb-repo)

• wget https://artifacts.elastic.co/downloads/kibana/kibana-7.10.1-amd64.deb
• dpkg -i kibana-7.10.1-amd64.deb
• /etc/kibana/kibana.yml 파일 수정
• elasticsearch와 kibana 연동

• sudo systemctl start kibana.service
• systemctl enable kibana.service
  
  • 부팅시 자동실행
• troubleshooting
  • firewall-cmd --permanent --zone=public --add-port=5601/tcp  //filewall 설정
  • firewall-cmd --reload 
  • firewall-cmd --list-ports
  • virtualbox나 vmware를 사용할 경우 http://localhost:5601로 접속이 안될 수 있다. 
    
    • ifconfig로 서버 IP확인
      • http://{서버IP}:5601 로 접속

ElasticSearch 설치

• • ELK는 JVM위에서 구동
  • 설치환경에 JDK가 설치되어 있어야 한다.
    • apt-get install openjdk-11-jre
    • apt-get install openjdk-11-jdk

• wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.1-amd64.deb
• dpkg -i elasticsearch-7.10.1-amd64.deb

• Elasticsearch 실행
• # systemctl enable elasticsearch.service (서비스활성화- 부팅시 자동 구동)
• # service elasticsearch start

systemctl enable name.service

PUT vs POST(idempotent)

idempotent = 실행을 여러번 반복해도 같은 결과를 보장받을 수 있는 특성을 말함.

• PUT(idempotent O)
  • 리소스의 생성과 수정
  • 요청시마다 같은 리소스를 반환
  • PUT /<index>/<type>/<id>
• POST(idempotent X)
  • 리소스 생성 역할
  • 요청시마다 새로운 리소스 생성
  • POST /<index>/<type>

• curl -XGET http://localhost:9200/classes?pretty

• curl -XPUT http://localhost:9200/classes

• curl -XDELETE http://localhost:9200/classes

• curl -XPOST http://localhost:9200/classes/class/2?pretty -H'Content-Type: application/json' -d' {"title":"Algorithm","Professor":"John"}'

• curl -XPOST http://localhost:9200/classes/class/4?pretty -H'Content-Type: application/json' -d @./test.json

• curl -XPOST http://localhost:9200/classes/class/3?pretty -update -H'Content-Type: application/json' -d' {"doc": {"Unit" : 1}}'

• curl -XPOST http://localhost:9200/_bulk --header 'content-type: application/json' --data-binary @classes.json

매핑테스트

• curl -XPUT localhost:9200/classes/class/_mapping?include_type_name=true -d @classesRating_mapping.json  -H 'Content-Type: application/json'

• curl -XGET localhost:9200/basketball/record/_search?pretty

• curl -XGET 'localhost:9200/basketball/record/_search?q=points:30&pretty'
• curl -XGET localhost:9200/basketball/record/_search?pretty --header 'content-type:application/json' -d '  { 

"query":{
"term":{"points":30}
}
}'

• curl -XGET localhost:9200/_search?pretty --data-binary @avg_points_aggs.json  --header 'content-type:application/json'
  
  • "points" : 20과  "points" : 30의 평균값 출력
• curl -XGET localhost:9200/_search?pretty --data-binary @max_points_aggs.json  --header 'content-type:application/json'
  
  • max값 출력가능
• curl -XGET localhost:9200/_search?pretty --data-binary @min_points_aggs.json  --header 'content-type:application/json'
  
  • min값 출력
• curl -XGET localhost:9200/_search?pretty --data-binary @sum_points_aggs.json  --header 'content-type:application/json'

• curl -XGET localhost:9200/_search?pretty --data-binary @stats_points_aggs.json  --header 'content-type:application/json'

• curl -XGET localhost:9200/_search?pretty --data-binary @terms_aggs.json --header 'content-type:application/json'
  • LA와 cicago를 group하여 출력

• curl -XGET localhost:9200/_search?pretty --data-binary @stats_by_team.json --header 'content-type:application/json'