一 DSL查询分类及基本语法

1.DSL Query的分类

Elasticsearch提供了基于JSON的DSL(Domain Specific Language)来定义查询。常见的查询类型包括：

  • 查询所有：查询出所有数据，一般测试试用。例如：match_all
  • 全文检索（full text）查询：利用分词器对用户输入内容分词，然后去倒排索引库中匹配。例如
    • match_query
    • multi_match_query
  • 精确查询：根据精确词条值查找数据，一般是查找keyword、数值、日期、boolean等类型字段。例如：
    • ids
    • range
    • term
  • 地理(geo)查询：根据经纬度查询。例如：
    • geo_distance
    • geo_bounding_box
  • 复合（compound）查询：复合查询可以将上述各种查询条件组合起来，合并查询条件。例如：
    • bool
    • function_score

2.查询所有

查询的基本语法如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
GET /indexName/_search
{
  "query": {
    "查询类型": {
      "查询条件": "条件值"
    }
  }
}
// 查询所有
GET /hotel/_search
{
  "query": {
    "match_all": {
    }
  }
}

3.全文检索查询

全文检索查询，会对用户输入内容分词，常用于搜索框搜索：

match查询：全文检索查询的一种，会对用户输入内容分词，然后去倒排索引库检索，语法：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
GET /indexName/_search
{
  "query": {
    "match": {
      "FIELD": "TEXT"
    }
  }
}
# match查询
GET /hotel/_search
{
  "query": {
    "match": {
      "all": "如家外滩"
    }
  }
}

multi_match查询：与match查询类似，只不过允许同时查询多个字段语法：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
GET /indexName/_search
{
  "query": {
    "multi_match": {
      "query": "TEXT"
      "FIELD": ["FIELD1", "FIELD2"]
    }
  }
}
# multi_match查询
GET /hotel/_search
{
  "query": {
    "multi_match": {
      "query": "外滩如家",
      "fields": ["brand", "name", "business"]
    }
  }
}

4.精确查询

精确查询一般是查找keyword、数值、日期、boolean等类型字段。所以不会对搜索条件分词。常见的有：

  • term：根据词条精确值查询
  • range：根据值的范围查询

语法如下：

term查询：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
GET /indexName/_search
{
  "query": {
    "term": {
      "FIELD"; {
        'value': "VALUE"
      }
    }
  }
}
# term查询
GET /hotel/_search
{
  "query": {
    "term": {
      "city": {
        "value": "上海"
      }
    }
  }
}

range查询

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
GET /indexName/_search
{
  "query": {
    "range": {
      "FIELD": {
        "gte": 10,
        "lte": 20
      }
    }
  }
}
# range查询
GET /hotel/_search
{
  "query": {
    "range": {
      "price": {
        "gte": 100,
        "lte": 300
      }
    }
  }
}

5. 地理查询

根据经纬度查询。常见的使用场景包括：

  • 携程：搜索我附近的酒店
  • 滴滴：搜索我附近的出租车
  • 微信：附近的人

种类：

  • geo_bounding_box：查询geo_point值落在某个矩形范围的所以文档
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
GET /index_name/_search
{
  "query": {
    "geo_bounding_box": {
      "FILED": {
        "top_left": {
          "lat": 31.1,
          "lon": 121.5
        },
        "bottom_right": {
          "lat": 30.9,
          "lon": 121.7
        }
      }
    }
  }
}
  • geo_distance：查询到指定中心点小于某个距离值的所有文档
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
GET /index_name/_search
{
  "query": {
    "geo_distance": {
      "distance": "15km",
      "FIELD": "31.21,121.5"
    }
  }
}
# distance查询
GET /hotel/_search
{
  "query": {
    "geo_distance": {
      "distance": "15km",
      "location": "31.21,121.5"
    }
  }
}

6. 复合查询

复合（compound）查询：复合查询可以将其它简单查询组合起来，实现更复杂的搜索逻辑，例如：

  • function_score：算分函数查询，可以控制文档相关性算分，控制文档排名。例如百度竞价

相关性算分：当我们利用match查询时，文档结果会根据与搜索词条的关联度打分（_score），返回结果时按照分值降序排列。

TF(词条频率) = 词条出现次数 / 文档中词条总数

TF-IDF算法

IDF(逆文档频率) = log(文档总数 / 包含词条的文档总数)

score = 求和（TF(词条频率)） * IDF (逆文档频率)

BM25: Score(Q,d) = 求和(log(1 + ((N - n + 0.5) / (n + 0.5)))) * f(i) / (f(i) + k1 * (1 - b + b * (dl/avgdl)))

使用function score query。可以修改文档的

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
GET /index_name/_search
{
  "query": {
    "function_score": {
        "query": {"match": {"all": "外滩"}}, # 原始查询条件，搜索文档并根据相关性打分（query score）
        "functions": [
          {
            "filter": {"term": {"id": "1"}}, # 过滤条件，复合条件的文档才会被重新算分
            "weight": 10 # 算分函数，算分函数的结果称为function score，将来会与query score运算，得到新算分，常见的算分函数有： 1 weight 给一个常量值，作为函数结果（function score）2.field_value_factor:用文档中的某个字段值作为函数结果 3.random_score:随机生成一个值，作为函数结果 4.script_score:自定义计算公式，公式结果作为函数结果 
          }  
        ],
        "boost_mode": "multiply" # 加权模式，定义function score与query score的运算方式，包括: 1. multiply:两者相乘。默认就是这个 2.replace:用function score替换query score。 3. 其它:sum.avg.max、min
      }
  }
}

案例： 给"如家"这个品牌的酒店排名靠前一些

把这个问题翻译一下，function score需要的三要素：

  • 1.哪些文档需要算分加权
    • 品牌为如家的酒店
  • 2.算分函数是什么？
    • weight就可以
  • 3.加权模式是什么？
    • 求和
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
GET /hotel/_search
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "all": "外滩"
        }
      },
      "functions": [
        {
          "filter": {
            "term": {
              "brand": "如家"
            }
          },
          "weight": 10
        }
      ],
      "boost_mode": "sum"
    }
  }
}

7. 复合查询 Boolean Query

布尔查询是一个或多个查询子句的组合。子查询的组合方式有：

  • must：必须匹配每个子查询，类似与
  • should：选择性匹配子查询，类似于或
  • must_not：必须不匹配，不参与算分，类似非‘
  • filter：必须匹配，不参与算分

案例： 利用bool查询实现功能

需求：搜索包含如家，价格不高于400，在坐标31.21，121.5周围10km范围内的酒店

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
GET /hotel/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "name": "如家"
          }
        }
      ],
      "must_not": [
        {
          "range": {
            "price": {
              "gt": 400
            }
          }
        }
      ],
      "filter": [
        {
          "geo_distance": {
            "distance": "10km",
            "location": {
              "lat": 31.21,
              "lon": 121.5
            }
          }
        }
      ]
    }
  }

2. 搜索结果处理

1. 排序

elasticsearch支持对搜索结果排序，默认是根据相关度算分(_score)来排序。可以排序字段类型有: keyword类型、数值类型、地理坐标类型、日期类型等。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
GET /indexName/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "FIELD": {
        "order": "desc"
      }
    }
  ]
}
GET /indexName/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "_geo_distance": {
        "FIELD": {
          "lat": 40,
          "lon": -70
        },
        "order": "asc",
        "unit": "km"
      }
    }
  ]
}

案例1： 对酒店数据按照用户评价降序排序，评价相同的按照价格升序排序

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# sort排序
GET /indexName/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "score": "desc"
    },
    {
      "score": "asc"
    }
  ]
}

案例2： 实现对酒店数据按照你的位置坐标距离升序排序

获取经纬度的方式: https://lbs.amap.com/demo/jsapi-v2/example/map/click-to-get-lnglat

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# 找到121.612282，31.034661周围的酒店，距离升序排序
GET /hotel/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "_geo_distance": {
        "location": {
          "lat": 31.034661,
          "lon": 121.612282
        },
        "order": "asc",
        "unit": "km"
      }
    }
  ]
}

2. 分页

elasticsearch默认情况下只返回top10的数据。而如果要查询更多数据就需要修改分页参数了。

elasticsearch中通过修改from、size参数来控制要返回的分页结果:

1
2
3
4
5
6
7
8
9
10
11
12
13
GET /indexName/_search
{
  "query": {
    "match_all": {}
  },
  "from": 990,
  "size": 10,
  "sort": [
    {
      "price": "asc"
    }
  ]
}

深度分页问题： ES是分布式的，所以会面临深度分页问题。例如按price排序后，获取from = 990， size = 10的数据：

  • 1.首先在每个数据分片上都排序并查询前1000条文档。
  • 2.然后将所有节点的结果聚合，在内存中重新排序选出前1000条文档。
  • 3.最后从这1000条中，选取从990开始的10条文档

深度分页

如果搜索页数过深，或者结果集( from + siz,)越大，对内存和CPU的消耗也越高。因此ES设定结果集查询的上限是10000

深度分页解决方案：

针对深度分页，ES提供了两种解决方案：

  • search after:分页时需要排序，原理是从上一次的排序值开始，查询下一页数据。官方推荐使用的方式
  • scroll:原理将排序数据形成快照，保存在内存。官方已经不推荐使用。

3.高亮

高亮：就是在搜索结果中把搜索关键字突出显示

原理是这样的：

  • 将搜索关键字用标签标记出来
  • 在页面中给标签添加CSS样式
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
GET /indexName/_search
{
  "query": {
    "match": {
      "FIELD": "TEXT"
    }
  },
  "highlight": {
    "fields": {
      "FILED": {
        "pre_tags": "<em>",
        "post_tags": "</em>"
      }
    }
  }
}
# 高亮查询，默认情况下，ES搜索字段必须与高亮字段一致
GET /hotel/_search
{
  "query": {
    "match": {
      "all": "如家"
    }
  },
  "highlight": {
    "fields": {
      "name": {
        "pre_tags": "<em>",
        "post_tags": "</em>",
        "require_field_match": "false"
      }
    }
  }
}

3. RestClient查询文档

1.快速入门

通过match_all来演示下基本API，先看请求DSL的组织：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
package cn.itcast.hotel;

import org.apache.http.HttpHost;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import java.io.IOException;

public class HotelSearchTest {
    private RestHighLevelClient client;

    @Test
    void testMatchAll() throws IOException {
        // 1. 准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2. 准备DSL
        request.source().query(QueryBuilders.matchAllQuery());
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);

        System.out.println(response);
    }

    @BeforeEach
    void setUp() {
        this.client = new RestHighLevelClient(RestClient.builder(
                HttpHost.create("http://192.168.127.132:9200")
//                集群可写多个
//                HttpHost.create("http://192.168.127.131:9200"),
//                HttpHost.create("http://192.168.127.131:9200")
        ));
    }

    @AfterEach
    void tearDown() throws IOException {
        this.client.close();
    }
}

我们通过match_all来演示下基本的API，再看结果的解析

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
package cn.itcast.hotel;

import cn.itcast.hotel.pojo.HotelDoc;
import com.alibaba.fastjson.JSON;
import org.apache.http.HttpHost;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import java.io.IOException;

public class HotelSearchTest {
    private RestHighLevelClient client;

    @Test
    void testMatchAll() throws IOException {
        // 1. 准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2. 准备DSL
        request.source().query(QueryBuilders.matchAllQuery());
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        // 4.解析响应
        SearchHits searchHits = response.getHits();
        // 4.1获取总条数
        long total = searchHits.getTotalHits().value;
        System.out.println("共搜索到" + total + "条数据");
        // 4.2 文档数组
        SearchHit[] hits = searchHits.getHits();
        // 4.3 遍历
        for (SearchHit hit : hits) {
            String json = hit.getSourceAsString();
            // 反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            System.out.println("hotelDoc = " + hotelDoc);
        }
        System.out.println(response);
    }

    @BeforeEach
    void setUp() {
        this.client = new RestHighLevelClient(RestClient.builder(
                HttpHost.create("http://192.168.127.132:9200")
//                集群可写多个
//                HttpHost.create("http://192.168.127.131:9200"),
//                HttpHost.create("http://192.168.127.131:9200")
        ));
    }

    @AfterEach
    void tearDown() throws IOException {
        this.client.close();
    }
}

RestAPI中其中构建DSL是通过HighLevelRestClient中的resource()来实现的，其中包含了查询、排序、分页、高亮等

2.全文检索查询

全文检索的match和multi_match查询与match_all的API基本一致。差别是查询条件，也就是query的部分。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
package cn.itcast.hotel;

import cn.itcast.hotel.pojo.HotelDoc;
import com.alibaba.fastjson.JSON;
import org.apache.http.HttpHost;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import java.io.IOException;

public class HotelSearchTest {
    private RestHighLevelClient client;

    @Test
    void testMatch() throws IOException {
        // 1. 准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2. 准备DSL
        request.source().query(QueryBuilders.matchQuery("all", "如家"));
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        // 4.解析响应
        SearchHits searchHits = response.getHits();
        // 4.1获取总条数
        long total = searchHits.getTotalHits().value;
        System.out.println("共搜索到" + total + "条数据");
        // 4.2 文档数组
        SearchHit[] hits = searchHits.getHits();
        // 4.3 遍历
        for (SearchHit hit : hits) {
            String json = hit.getSourceAsString();
            // 反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            System.out.println("hotelDoc = " + hotelDoc);
        }
    }

    @BeforeEach
    void setUp() {
        this.client = new RestHighLevelClient(RestClient.builder(
                HttpHost.create("http://192.168.127.132:9200")
//                集群可写多个
//                HttpHost.create("http://192.168.127.131:9200"),
//                HttpHost.create("http://192.168.127.131:9200")
        ));
    }

    @AfterEach
    void tearDown() throws IOException {
        this.client.close();
    }
}

3.精确查询

精确查询常见的有term查询和range查询，同样利用QueryBuilders实现：

1
2
3
4
5
//词条查询
QueryBuilders.termQuery ( "city"，"杭州");
//范围查询
QueryBuilders.rangeQuery( "price").gte(100).lte( 150);

4.复合查询

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
//创建布尔查询
BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();s
//添加must条件
boolQuery.must(QueryBuilders.termQuery("city","杭州"))boolQuery.filter(QueryBuilders.rangeQuery("price").lte(250));
//添加filter条件
boolQuery.filter(QueryBuilders.rangeQuery("price").lte(250));


package cn.itcast.hotel;

import cn.itcast.hotel.pojo.HotelDoc;
import com.alibaba.fastjson.JSON;
import org.apache.http.HttpHost;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import java.io.IOException;

public class HotelSearchTest {
    private RestHighLevelClient client;

    @Test
    void testBool() throws IOException {
        // 1. 准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2. 准备DSL
        // 2.1 准备BooleanQuery
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        // 2.2 添加 term
        boolQuery.must(QueryBuilders.termQuery("city", "杭州"));
        // 2.3 添加 ranger
        boolQuery.filter(QueryBuilders.rangeQuery("price").lte(250));

        request.source().query();
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        // 4.解析响应
        SearchHits searchHits = response.getHits();
        handleResponse(searchHits);
    }

    private void handleResponse(SearchHits searchHits) {
        // 4.1获取总条数
        long total = searchHits.getTotalHits().value;
        System.out.println("共搜索到" + total + "条数据");
        // 4.2 文档数组
        SearchHit[] hits = searchHits.getHits();
        // 4.3 遍历
        for (SearchHit hit : hits) {
            String json = hit.getSourceAsString();
            // 反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            System.out.println("hotelDoc = " + hotelDoc);
        }
    }



    @BeforeEach
    void setUp() {
        this.client = new RestHighLevelClient(RestClient.builder(
                HttpHost.create("http://192.168.127.132:9200")
//                集群可写多个
//                HttpHost.create("http://192.168.127.131:9200"),
//                HttpHost.create("http://192.168.127.131:9200")
        ));
    }

    @AfterEach
    void tearDown() throws IOException {
        this.client.close();
    }
}

5.排序和分页

搜索结果的排序和分页是与query同级的参数，对应的API如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
// 查询
request.source( ).query(QueryBuilders.matchAllQuery());
//分页
request.source() .from(0 ).size(5);
/价格排序
request.source().sort("price",SortOrder.ASC);

package cn.itcast.hotel;

import cn.itcast.hotel.pojo.HotelDoc;
import com.alibaba.fastjson.JSON;
import org.apache.http.HttpHost;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortOrder;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import java.io.IOException;

public class HotelSearchTest {
    private RestHighLevelClient client;

    @Test
    void testPageAndSort() throws IOException {
        // 页码 ，每页大小
        int page = 1, size = 5;
        // 1. 准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2. 准备DSL
        // 2.1 准备Query
        request.source().query(QueryBuilders.matchAllQuery());
        // 2.2 排序 sort
        request.source().sort("price", SortOrder.ASC);
        // 2.3 分页 from ， size
        request.source().from((page - 1) * size).size(5);
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        // 4.解析响应
        SearchHits searchHits = response.getHits();
        handleResponse(searchHits);
    }

    private void handleResponse(SearchHits searchHits) {
        // 4.1获取总条数
        long total = searchHits.getTotalHits().value;
        System.out.println("共搜索到" + total + "条数据");
        // 4.2 文档数组
        SearchHit[] hits = searchHits.getHits();
        // 4.3 遍历
        for (SearchHit hit : hits) {
            String json = hit.getSourceAsString();
            // 反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            System.out.println("hotelDoc = " + hotelDoc);
        }
    }



    @BeforeEach
    void setUp() {
        this.client = new RestHighLevelClient(RestClient.builder(
                HttpHost.create("http://192.168.127.132:9200")
//                集群可写多个
//                HttpHost.create("http://192.168.127.131:9200"),
//                HttpHost.create("http://192.168.127.131:9200")
        ));
    }

    @AfterEach
    void tearDown() throws IOException {
        this.client.close();
    }
}

6. 高亮

高亮API包括请求DSL构建和结果解析两部分。我们先看请求的DSL构建:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
request.source ( ).highlighter( new HighlightBuilder(){
    .field ( "name")
    //是否需要与查询字段匹配
    .requireFieldMatch(false)
);
//获取source
HotelDoc hotelDoc = JSON.parseObject(hit.getSourceAsString()，HotelDoc.class);
if ( !collectionUtils.isEmpty(highlightFields) ) {
    //获取高亮字段结果
    HighlightField highlightField = highlightFields.get( "name");
    if (highlightField != null) {
        //取出高亮结果数组中的第一个，就是酒店名称
        string name = highlightField.getFragments()[0].string( );
        hotelDoc.setName(name);
    }
}    
// 示例代码    
package cn.itcast.hotel;

import cn.itcast.hotel.pojo.HotelDoc;
import com.alibaba.fastjson.JSON;
import org.apache.http.HttpHost;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.elasticsearch.search.sort.SortOrder;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.util.CollectionUtils;

import java.io.IOException;
import java.util.Map;

public class HotelSearchTest {
    private RestHighLevelClient client;
    
    @Test
    void testHighlight() throws IOException {
        // 1. 准备Request
        SearchRequest request = new SearchRequest("hotel");
        // 2. 准备DSL
        // 2.1 准备Query
        request.source().query(QueryBuilders.matchQuery("all", "如家"));
        // 2.2 高亮
        request.source().highlighter(new HighlightBuilder().field("name").requireFieldMatch(false));
        // 3.发送请求
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        // 4.解析响应
        SearchHits searchHits = response.getHits();
        handleResponse(searchHits);
    }

    private void handleResponse(SearchHits searchHits) {
        // 4.1获取总条数
        long total = searchHits.getTotalHits().value;
        System.out.println("共搜索到" + total + "条数据");
        // 4.2 文档数组
        SearchHit[] hits = searchHits.getHits();
        // 4.3 遍历
        for (SearchHit hit : hits) {
            String json = hit.getSourceAsString();
            // 反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            // 获取高亮结果
            Map<String, HighlightField> highlightFields = hit.getHighlightFields();
            if (!CollectionUtils.isEmpty(highlightFields)){
                // 根据字段名称获取高亮结果
                HighlightField highlightField = highlightFields.get("name");
                if (highlightField != null){
                    // 获取高亮值
                    String name = highlightField.getFragments()[0].string();
                    // 覆盖非高亮结果
                    hotelDoc.setName(name);
                }
            }
            System.out.println("hotelDoc = " + hotelDoc);
        }
    }



    @BeforeEach
    void setUp() {
        this.client = new RestHighLevelClient(RestClient.builder(
                HttpHost.create("http://192.168.127.132:9200")
//                集群可写多个
//                HttpHost.create("http://192.168.127.131:9200"),
//                HttpHost.create("http://192.168.127.131:9200")
        ));
    }

    @AfterEach
    void tearDown() throws IOException {
        this.client.close();
    }
}

五 旅游案例

案例一： 实现旅游的酒店搜索功能，完成关键字搜索和分页

实现关键字搜索功能，实现步骤如下：

  • 1.定义实体类，接收前端请求
  • 2.定义controller接口，接收页面请求，调用IHotelService的search方法
  • 3.定义定义lHotelService中的search方法，利用match查询实现根据关键字搜索酒店信息

步骤1：定义类，接收 前端请求参数

1
2
3
4
5
6
7
8
9
10
11
12
package cn.itcast.hotel.pojo;

import lombok.Data;

@Data
public class RequestParams {
    private String key;
    private Integer page;
    private Integer size;
    private String sortBy;
}

步骤2：定义controller接口，接收前端请求

定义一个HotelController，声明查询接口，满足下列要求:
请求方式:Post

  • 请求路径: /hotel/list
  • 请求参数:对象，类型为RequestParam
  • 返回值:PageResult，包含两个属性：
    • Long total:总条数
    • List<HotelDoc> hotels:酒店数据
1
2
3
4
5
6
7
8
9
10
11
12
package cn.itcast.hotel.pojo;

import lombok.Data;

import java.util.List;

@Data
public class PageResult {
    private Long total;
    private List<HotelDoc> hotels;
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
package cn.itcast.hotel.web;

import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/hotel")
public class HotelController {

    @Autowired
    private IHotelService hotelService;
    @PostMapping("/list")
    public PageResult search(@RequestBody RequestParams params){
        return hotelService.search(params);

    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
package cn.itcast.hotel.service.impl;

import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService {
   @Autowired
   private RestHighLevelClient client;

    @Override
    public PageResult search(RequestParams params) {
        try {
            // 1. 准备Request
            SearchRequest request = new SearchRequest("hotel");
            // 2. 准备DSL
            // 2.1 关键字搜索
            String key = params.getKey();
            if (key == null || "".equals(key)){
                request.source().query(QueryBuilders.matchAllQuery());
            }else {
                request.source().query(QueryBuilders.matchQuery("all", key));
            }
            // 2.2 分页
            int page = params.getPage();
            int size = params.getSize();
            request.source().from((page - 1) * size).size(size);
            // 3.发送请求
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            // 4.解析响应
            SearchHits searchHits = response.getHits();
            return handleResponse(searchHits);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

    }

    private PageResult handleResponse(SearchHits searchHits) {
        // 4.1获取总条数
        long total = searchHits.getTotalHits().value;
        // 4.2 文档数组
        SearchHit[] hits = searchHits.getHits();
        // 4.3 遍历
        List<HotelDoc> hotels = new ArrayList<>();
        for (SearchHit hit : hits) {
            String json = hit.getSourceAsString();
            // 反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            hotels.add(hotelDoc);
        }
        return new PageResult(total, hotels);
    }
}

记得注入RestHighLevelClient

案例二：添加品牌、城市、星级、价格等过滤功能

步骤：

  • 1．修改RequestParams类，添加prand、city、starName、minPrice、maxPrice等参数
  • 2．修改search方法的实现，在关键字搜索时，如果brand等参数存在，对其做过滤

步骤1：拓展IUserService的search方法的参数列表

修改RequestParams类，接收所有参数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
package cn.itcast.hotel.pojo;

import lombok.Data;

@Data
public class RequestParams {
    private String key;
    private Integer page;
    private Integer size;
    private String sortBy;
    private String city;
    private String brand;
    private String starName;
    private Integer minPrice;
    private Integer maxPrice;
}

步骤2：修改search方法，在match查询基础上添加过滤条件

过滤条件包括：

  • city精确匹配
  • brand精确匹配
  • starName精确匹配
  • price范围过滤

注意事项:

  • 多个条件之间是AND关系，组合多条件用BooleanQuery
  • 参数存在才需要过滤，做好非空判断

案例三：我附近的酒店

前端页面点击定位后，会将所在的位置发送到后台
，我们要根据这个坐标，将酒店结果按照到这个点的距离升序排序。

实现思路：

  • 修改RequestParams参数，接收location字段
  • 修改search方法业务逻辑，如果location有值，添加根据geo_distance排序的功能
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
package cn.itcast.hotel.pojo;

import lombok.Data;

@Data
public class RequestParams {
    private String key;
    private Integer page;
    private Integer size;
    private String sortBy;
    private String city;
    private String brand;
    private String starName;
    private Integer minPrice;
    private Integer maxPrice;
    private String location;
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
package cn.itcast.hotel.service.impl;

import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.geo.GeoPoint;
import org.elasticsearch.common.unit.DistanceUnit;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService {
   @Autowired
   private RestHighLevelClient client;

    @Override
    public PageResult search(RequestParams params) {
        try {
            // 1. 准备Request
            SearchRequest request = new SearchRequest("hotel");
            // 2. 准备DSL
            // 2.1 关键字搜索
            buildBasicQuery(params, request);
            // 2.2 分页
            int page = params.getPage();
            int size = params.getSize();
            request.source().from((page - 1) * size).size(size);
            // 2.3排序
            String location = params.getLocation();
            if (location != null && "".equals(location)){
                request.source().sort(SortBuilders
                        .geoDistanceSort("location", new GeoPoint(location))
                        .order(SortOrder.ASC)
                        .unit(DistanceUnit.KILOMETERS)
                );
            }
            // 3.发送请求
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            // 4.解析响应
            SearchHits searchHits = response.getHits();
            return handleResponse(searchHits);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

    }

    private void buildBasicQuery(RequestParams params, SearchRequest request) {
        // 构建BooleanQuery
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        // 关键字搜索
        String key = params.getKey();
        if (key == null || "".equals(key)){
            boolQuery.must(QueryBuilders.matchAllQuery());
        }else {
            boolQuery.must(QueryBuilders.matchQuery("all", key));
        }
        // 条件过滤
        // 城市条件
        if (params.getCity() != null && !params.getCity().equals("")){
            boolQuery.filter(QueryBuilders.termQuery("city", params.getCity()));
        }
        // 品牌条件
        if (params.getBrand() != null && !params.getBrand().equals("")){
            boolQuery.filter(QueryBuilders.termQuery("brand", params.getBrand()));
        }
        // 星级条件
        if (params.getStarName() != null && !params.getStarName().equals("")){
            boolQuery.filter(QueryBuilders.termQuery("starName", params.getStarName()));
        }
        // 价格条件
        if (params.getMinPrice() != null && params.getMaxPrice() != null){
            boolQuery.filter(QueryBuilders
                    .rangeQuery("price").gte(params.getMinPrice()).lte(params.getMaxPrice()));
        }
        request.source().query(boolQuery);
    }

    private PageResult handleResponse(SearchHits searchHits) {
        // 4.1获取总条数
        long total = searchHits.getTotalHits().value;
        // 4.2 文档数组
        SearchHit[] hits = searchHits.getHits();
        // 4.3 遍历
        List<HotelDoc> hotels = new ArrayList<>();
        for (SearchHit hit : hits) {
            String json = hit.getSourceAsString();
            // 反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            // 获取排序值
            Object[] sortValues = hit.getSortValues();
            if (sortValues.length > 0){
                Object sortValue = sortValues[0];
                hotelDoc.setDistance(sortValue);
            }
            hotels.add(hotelDoc);
        }
        return new PageResult(total, hotels);
    }
}


案例四：让指定的酒店在搜索结果中排名置顶

我们给需要置顶的酒店文档添加一个标记。然后利用function score给带有标记的文档增加权重。

实现步骤分析:

  • 1.给HotelDoc类添加isAD字段，Boolean类型
  • 2.挑选几个你喜欢的酒店，给它的文档数据添加isAD字段，值为true
  • 3．修改search方法，添加function score功能，给isAD值为true的酒店增加权重
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
package cn.itcast.hotel.pojo;

import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@NoArgsConstructor
public class HotelDoc {
    private Long id;
    private String name;
    private String address;
    private Integer price;
    private Integer score;
    private String brand;
    private String city;
    private String starName;
    private String business;
    private String location;
    private String pic;
    private Object distance;
    private Boolean isAD;

    public HotelDoc(Hotel hotel) {
        this.id = hotel.getId();
        this.name = hotel.getName();
        this.address = hotel.getAddress();
        this.price = hotel.getPrice();
        this.score = hotel.getScore();
        this.brand = hotel.getBrand();
        this.city = hotel.getCity();
        this.starName = hotel.getStarName();
        this.business = hotel.getBusiness();
        this.location = hotel.getLatitude() + ", " + hotel.getLongitude();
        this.pic = hotel.getPic();
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
package cn.itcast.hotel.service.impl;

import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.geo.GeoPoint;
import org.elasticsearch.common.unit.DistanceUnit;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.functionscore.FunctionScoreQueryBuilder;
import org.elasticsearch.index.query.functionscore.ScoreFunctionBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService {
   @Autowired
   private RestHighLevelClient client;

    @Override
    public PageResult search(RequestParams params) {
        try {
            // 1. 准备Request
            SearchRequest request = new SearchRequest("hotel");
            // 2. 准备DSL
            // 2.1 关键字搜索
            buildBasicQuery(params, request);
            // 2.2 分页
            int page = params.getPage();
            int size = params.getSize();
            request.source().from((page - 1) * size).size(size);
            // 2.3排序
            String location = params.getLocation();
            if (location != null && "".equals(location)){
                request.source().sort(SortBuilders
                        .geoDistanceSort("location", new GeoPoint(location))
                        .order(SortOrder.ASC)
                        .unit(DistanceUnit.KILOMETERS)
                );
            }
            // 3.发送请求
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            // 4.解析响应
            SearchHits searchHits = response.getHits();
            return handleResponse(searchHits);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

    }

    private void buildBasicQuery(RequestParams params, SearchRequest request) {
        // 构建BooleanQuery
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        // 关键字搜索
        String key = params.getKey();
        if (key == null || "".equals(key)){
            boolQuery.must(QueryBuilders.matchAllQuery());
        }else {
            boolQuery.must(QueryBuilders.matchQuery("all", key));
        }
        // 条件过滤
        // 城市条件
        if (params.getCity() != null && !params.getCity().equals("")){
            boolQuery.filter(QueryBuilders.termQuery("city", params.getCity()));
        }
        // 品牌条件
        if (params.getBrand() != null && !params.getBrand().equals("")){
            boolQuery.filter(QueryBuilders.termQuery("brand", params.getBrand()));
        }
        // 星级条件
        if (params.getStarName() != null && !params.getStarName().equals("")){
            boolQuery.filter(QueryBuilders.termQuery("starName", params.getStarName()));
        }
        // 价格条件
        if (params.getMinPrice() != null && params.getMaxPrice() != null){
            boolQuery.filter(QueryBuilders
                    .rangeQuery("price").gte(params.getMinPrice()).lte(params.getMaxPrice()));
        }
        // 2. 算分控制
        FunctionScoreQueryBuilder functionScoreQueryBuilder =
                QueryBuilders.functionScoreQuery(
                        // 原生查询 相关性算分
                        boolQuery,
                        // function score 数组
                        new FunctionScoreQueryBuilder.FilterFunctionBuilder[]{
//                                其中一个function score
                                new FunctionScoreQueryBuilder.FilterFunctionBuilder(
                                        // 过滤条件
                                        QueryBuilders.termQuery("AD", true),
                                        // 算分函数
                                        ScoreFunctionBuilders.weightFactorFunction(10)
                                )
                        });
        request.source().query(functionScoreQueryBuilder);
    }

    private PageResult handleResponse(SearchHits searchHits) {
        // 4.1获取总条数
        long total = searchHits.getTotalHits().value;
        // 4.2 文档数组
        SearchHit[] hits = searchHits.getHits();
        // 4.3 遍历
        List<HotelDoc> hotels = new ArrayList<>();
        for (SearchHit hit : hits) {
            String json = hit.getSourceAsString();
            // 反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
            // 获取排序值
            Object[] sortValues = hit.getSortValues();
            if (sortValues.length > 0){
                Object sortValue = sortValues[0];
                hotelDoc.setDistance(sortValue);
            }
            hotels.add(hotelDoc);
        }
        return new PageResult(total, hotels);
    }
}