需求分析

针对ST聊天内容无法搜索,使用ElasticSearch对聊天内容进行采集分析入库,提供对聊天内容的搜索功能

系统设计

数据采集入库设计

数据搜索设计

索引库设计

JDom解析

JDom类似于python中的beautifulSoup。一个强大的DOM解析器。可以针对XML和HTML的标签式语言进行解析。

ES-Client初始化

指定host与端口,并通过sniff嗅探。通过轮询的机制访问集群中所有节点。

public static void initClient(){
ConfigReader configReader=new ConfigReader();
String clusterName=configReader.getConfig("clusterName");
String ip=configReader.getConfig("ip");
Settings settings = Settings.builder()
.put("cluster.name", clusterName)
.put("client.transport.sniff", true).build();
try {
client = new PreBuiltTransportClient(settings)
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(ip), 9300));
} catch (UnknownHostException e) {
e.printStackTrace();
}
}

构建新增索引request请求

for (STMessage stMessage:list){
String UUIDString = UUID.randomUUID().toString().replace("-", "");
Map<String,Object> json=new HashMap<String, Object>();
json.put("context",stMessage.getContext());
json.put("time",stMessage.getTime());
json.put("user",stMessage.getUser());
json.put("url",stMessage.getUrl());
IndexResponse response = client.prepareIndex(index,type,UUIDString)
.setSource(json)
.get();
if (!"CREATED".equals(response.getResult().toString())){
success=false;
}
}

构建查询索引request请求

public List<STMessage> termSearch(TransportClient client,String field,String value){
List<STMessage> stMessageList=new ArrayList<STMessage>();
ConfigReader configReader=new ConfigReader();
String indexName=configReader.getConfig("index");
String typeName=configReader.getConfig("type");
SearchResponse searchResponse = client.prepareSearch(indexName)
.setTypes(typeName)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(QueryBuilders.termQuery(field, value)) // Query
.setFrom(0).setSize(60).setExplain(true)
.get();
SearchHits hits = searchResponse.getHits();
for (SearchHit searchHit:hits){
STMessage stMessage=new STMessage();
Map<String, Object> sourceAsMap = searchHit.sourceAsMap();
stMessage.setContext(String.valueOf(sourceAsMap.get("context")));
stMessage.setTime(String.valueOf(sourceAsMap.get("time")));
stMessage.setUser(String.valueOf(sourceAsMap.get("user")));
stMessage.setUrl(String.valueOf(sourceAsMap.get("url")));
stMessageList.add(stMessage);
}
return stMessageList;
}