GrayLog使用HTTP JSONPath方式调用微步在线云API识别威胁IP

需求：

针对GrayLog的安全告警日志中的一些攻击IP，经常需要手工去微步在线情报社区去查恶意IP，为了提高效率，探索是否可以对IP进行自动化关联查询

(图片点击放大查看)

思路：

1、GrayLog的LookupTable有HTTP JSONPATH这种方式，可以研究一下怎么使用

2、翻阅微步在线API接口文档以及一些常用的IP地址库接口API

3、结合GrayLog强大的可定制化的功能，探索是否可以实现想要的效果

解决过程与步骤：

1、微步在线API接口文档

(图片点击放大查看)

(图片点击放大查看)

(图片点击放大查看)

2、curl命令测试

curl -v -X GET 'https://api.threatbook.cn/v3/scene/ip_reputation?apikey=API_KEY&resource=37.59.54.205'

(图片点击放大查看)

yum install jq -y
curl  -X GET 'https://api.threatbook.cn/v3/scene/ip_reputation?apikey=API_KEY&resource=37.59.54.205'| jq 

(图片点击放大查看)

3、GrayLog配置Lookup Tables

https://api.threatbook.cn/v3/scene/ip_reputation?apikey=API_KEY&resource=${key}

(图片点击放大查看)

(图片点击放大查看)

(图片点击放大查看)

{
  "data": {
    "37.59.54.205": {
      "severity": "low",
      "judgments": [
        "Scanner",
        "IDC"
      ],
      "tags_classes": [],
      "basic": {
        "carrier": "OVH SAS",
        "location": {
          "country": "France",
          "province": "Ile-de-France",
          "city": "Paris",
          "lng": "2.345101",
          "lat": "48.848483",
          "country_code": "FR"
        }
      },
      "asn": {
        "rank": 2,
        "info": "OVH, FR",
        "number": 16276
      },
      "scene": "",
      "confidence_level": "low",
      "is_malicious": true,
      "update_time": "2020-11-25 09:11:44"
    }
  },
  "response_code": 0,
  "verbose_msg": "OK"
}

(图片点击放大查看)

(图片点击放大查看)

IPv4 归属地，经纬度查询可以使用如下API接口

接口地址：https://api.ipplus360.com/ip/geo/v1/city/
请求方式：HTTPS GET/POST
请求示例：https://api.ipplus360.com/ip/geo/v1/city/?key=您申请的key&ip=您需要查询的ip&coordsys=WGS84

(图片点击放大查看)

(图片点击放大查看)

最后针对以上API接口配置的LookupTable如下

(图片点击放大查看)

4、可以针对有公网IP字段调用以上接口并将查询结果保存为日志

例如对Linux服务器暴力破解攻击IP为例

例如将Linux服务器接入到GrayLog后，使用正则表达式提取器提取出暴力破解IP的ssh_login_failed_ip字段

Failed password for .* from(.+?)\s.?

(图片点击放大查看)

然后使用LookupTable提取器，将IP经纬度结果转成字段

ip_geo_latitude Lookup Table
Trying to extract data from ssh_login_failed_ip into ssh_login_failed_ip_geo_latitude

ip_geo_longitude Lookup Table
Trying to extract data from ssh_login_failed_ip into ssh_login_failed_ip_geo_longitude

当然也可以调用IP威胁情报的LookupTable

ip_geo_longitude Lookup Table
Trying to extract data from ssh_login_failed_ip into ssh_login_failed_ip_geo_longitude

(图片点击放大查看)

5、Pipeline配置

使用Pipeline拼接出经纬度地址位置字段

rule "GeoIP lookup: ssh_login_failed_ip"
when
  has_field("ssh_login_failed_ip_geo_latitude") and has_field("ssh_login_failed_ip_geo_longitude")
then
    let ip_geo_latitude = to_string($message.ssh_login_failed_ip_geo_latitude);
    let ip_geo_longitude = to_string($message.ssh_login_failed_ip_geo_longitude);
    let ip_location_temp1 =  concat(to_string(ip_geo_latitude),",");
    let ip_location_temp2 =  concat(ip_location_temp1, ip_geo_longitude);
    set_field("ssh_login_failed_ip_location", ip_location_temp2);
end

(图片点击放大查看)

使用Pipeline配置生成IP威胁情报查询结果字段ssh_login_failed_ip_threatcheck_result

ip_threatcheck_result
rule "IPThreatChecklookup: ssh_login_failed_ip"
when
  has_field("ssh_login_failed_ip")
then
    let ip_threatcheck_result = lookup("ipthreatcheck",to_string($message.ssh_login_failed_ip));
    debug (ip_threatcheck_result) ;
    set_field("ssh_login_failed_ip_threatcheck_result", ip_threatcheck_result);
end

(图片点击放大查看)

(图片点击放大查看)

6、配置DashBoard显示攻击IP的经纬度

最终的效果

(图片点击放大查看)

(图片点击放大查看)

(图片点击放大查看)

如上图所示：Zombie关键字搜索可以判定该攻击IP为僵尸傀儡机

地理位置为荷兰

Tips:

附上提取器的相关语法配置文件

{
  "extractors": [
    {
      "title": "ip_geo_latitude",
      "extractor_type": "lookup_table",
      "converters": [],
      "order": 1,
      "cursor_strategy": "copy",
      "source_field": "ssh_login_failed_ip",
      "target_field": "ssh_login_failed_ip_geo_latitude",
      "extractor_config": {
        "lookup_table_name": "ipplus360_lat"
      },
      "condition_type": "none",
      "condition_value": ""
    },
    {
      "title": "ip_geo_longitude",
      "extractor_type": "lookup_table",
      "converters": [],
      "order": 2,
      "cursor_strategy": "copy",
      "source_field": "ssh_login_failed_ip",
      "target_field": "ssh_login_failed_ip_geo_longitude",
      "extractor_config": {
        "lookup_table_name": "ipplus360_lng"
      },
      "condition_type": "none",
      "condition_value": ""
    },
    {
      "title": "ip_threatcheck",
      "extractor_type": "lookup_table",
      "converters": [],
      "order": 3,
      "cursor_strategy": "copy",
      "source_field": "ssh_login_failed_ip",
      "target_field": "ssh_login_failed_ip_threatcheck",
      "extractor_config": {
        "lookup_table_name": "ipthreatcheck"
      },
      "condition_type": "none",
      "condition_value": ""
    },
    {
      "title": "json_extractor",
      "extractor_type": "json",
      "converters": [],
      "order": 4,
      "cursor_strategy": "copy",
      "source_field": "ssh_login_failed_ip_threatcheck_result",
      "target_field": "",
      "extractor_config": {
        "flatten": false,
        "list_separator": ", ",
        "kv_separator": "=",
        "key_prefix": "ip_threatcheck_result_",
        "key_separator": "_",
        "replace_key_whitespace": false,
        "key_whitespace_replacement": "_"
      },
      "condition_type": "none",
      "condition_value": ""
    },
    {
      "title": "ssh_login_failed_ip_extractor",
      "extractor_type": "regex",
      "converters": [],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "ssh_login_failed_ip",
      "extractor_config": {
        "regex_value": "Failed password for .* from(.+?)\\s.?"
      },
      "condition_type": "string",
      "condition_value": "Failed password "
    }
  ],
  "version": "4.2.10"
}