sphinx配置
sphinx是俄罗斯人开发的一个搜索引擎,基于c++编写,具有强大的检索能力,本身支持中文单个字符的检索,中文分词需要额外的插件Coreseek,但该插件已很久未更新,github用户eric1688基于sphinx 2.2.9版本改写了可以较好支持中文分词的sphinx for chinese,本文配置均基于此版本,下载地址在此,向作者致敬。
另:如果是别的系统,建议使用elastic search开发,对中文分词的支持更完善一些,社区更为活跃,但需求不复杂的话,sphinx已足够,而且配置相比更方便些。
安装sphinx
本文基于ubuntu 18.04安装sphinx,从上述地址下载压缩包,以下示例大部分基于作者的说明
下载解压
$ git clone https://github.com/eric1688/sphinx
$ cd sphinx
编译(假设安装到/usr/local/sphinx目录,下同)
安装gcc 和 mysql
sudo apt update
sudo apt install build-essential
sudo apt install mysql-server
编译安装sphinx
$ ./configure --prefix=/usr/local/sphinx --prefix 指定安装路径 --with-mysql 编译mysql支持 --with-pgsql 编译pgsql支持
$ make
$ make install
安装完成后,sphinx目录下应有bin、etc、share三个子目录,新建data和log两个子目录
配置中文支持
$ tar -xvf xdict_1.1.tar.gz $ /usr/local/sphinx/bin/mkdict xdict_1.1.txt xdict
从xdict_1.1.txt生成xdict文件,xdict_1.1.txt文件可以根据需要进行修改
$ cp xdict /usr/local/sphinx/etc/
配置sphinx.conf
sphinx.conf是最关键的一部分,sphinx的运作主要基于该文件。以下是本人的配置文件,结合phpcms数据库进行配置,该文件存放于bin子目录中
# 主索引数据源
source main
{
type = mysql
sql_host = 192.168.91.1 #phpcms所使用数据库地址
sql_user = sphinx #用户名
sql_pass = sphinx #密码
sql_db = phpcms_v9 #phpcms数据库名字
sql_port = 3306 #数据库端口
sql_query_pre = SET NAMES utf8
sql_query_pre = REPLACE INTO v9_sphinx_counter SELECT 1, MAX(searchid) FROM v9_search
sql_query = SELECT searchid, adddate, siteid, typeid, id as news_id, data FROM v9_search WHERE searchid>=$start AND searchid<=$end
sql_query_range = SELECT 1,max_doc_id FROM v9_sphinx_counter WHERE counter_id=1
sql_range_step = 5000
sql_attr_uint = typeid
sql_attr_uint = siteid
sql_attr_uint = news_id
sql_attr_timestamp = adddate
sql_query_info = SELECT * FROM v9_search WHERE searchid=$id
}
#增量索引数据源
source delta
{
type = mysql
sql_host = 192.168.91.1 #phpcms所使用数据库地址
sql_user = sphinx #用户名
sql_pass = sphinx #密码
sql_db = phpcms_v9 #phpcms数据库名字
sql_port = 3306 #数据库端口
sql_query_pre = SET NAMES utf8
sql_query_pre = REPLACE INTO v9_sphinx_counter SELECT 1, MAX(searchid) FROM v9_search
sql_query = SELECT searchid, adddate, siteid, typeid, id as news_id, data FROM v9_search WHERE searchid>=$start AND searchid<=$end
sql_query_range = SELECT 1,max_doc_id FROM v9_sphinx_counter WHERE counter_id=1
sql_range_step = 5000
sql_attr_uint = typeid
sql_attr_uint = siteid
sql_attr_uint = news_id
sql_attr_timestamp = adddate
sql_query_info = SELECT * FROM v9_search WHERE searchid=$id
}
#主索引
index main
{
source = main
path = /usr/local/sphinx/data/main
charset_type = utf-8
chinese_dictionary = /usr/local/sphinx/etc/xdict
}
#增量索引
index delta
{
source = delta
path = /usr/local/sphinx/data/delta
charset_type = utf-8
chinese_dictionary = /usr/local/sphinx/etc/xdict
}
indexer
{
mem_limit = 128M
}
searchd
{
listen = 9312
listen = 9306:mysql41 #监听端口,配置完成后可用于测试
log = /usr/local/sphinx/log/searchd.log
query_log = /usr/local/sphinx/log/query.log
read_timeout = 5
max_children = 30
pid_file = /usr/local/sphinx/log/searchd.pid
seamless_rotate = 1
preopen_indexes = 1
unlink_old = 1
binlog_path = /usr/local/sphinx/data
}
建立索引文件,并开启服务
# 以下命令执行的当前目录均为/usr/local/sphinx
$ ./indexer --all
$ ./searchd
测试
# 假设sphinx所在服务器ip为192.168.91.130
$ mysql -h192.168.91.130 -P9306
mysql> select * from main;
+------+------------+--------+--------+---------+
| id | adddate | siteid | typeid | news_id |
+------+------------+--------+--------+---------+
| 941 | 1585707168 | 1 | 1 | 231 |
| 942 | 1585707168 | 1 | 1 | 232 |
| 943 | 1585707168 | 1 | 1 | 233 |
| 944 | 1585707168 | 1 | 1 | 234 |
| 945 | 1585707168 | 1 | 1 | 235 |
| 946 | 1585707168 | 1 | 1 | 236 |
| 947 | 1585707168 | 1 | 1 | 237 |
| 948 | 1585707168 | 1 | 1 | 238 |
| 949 | 1585707168 | 1 | 1 | 239 |
| 950 | 1585707168 | 1 | 1 | 240 |
| 951 | 1585707168 | 1 | 1 | 241 |
| 952 | 1585707168 | 1 | 1 | 242 |
| 953 | 1585707168 | 1 | 1 | 243 |
| 954 | 1585707168 | 1 | 1 | 244 |
| 955 | 1585707168 | 1 | 1 | 245 |
| 956 | 1585707168 | 1 | 1 | 246 |
| 957 | 1585707168 | 1 | 1 | 247 |
| 958 | 1585707168 | 1 | 1 | 248 |
+------+------------+--------+--------+---------+
20 rows in set (0.00 sec)
phpcms相关配置和部分文件修改(todo)
phpcms(版本:phpcmsv9 9.6.3)集成了sphinx的接口,也有相关的文档,但对应的sphinx版本较老,因此要正常使用,需要做一定的修改
后台配置
模块->全站搜索->模块配置
基本配置:“是否启用全站搜索”选择“是”
sphinx全文索引配置:选择启用,服务器主机地址:192.168.91.130 端口号:9312
相关代码修改
phpcms>modules>search>index.php
......
//如果开启sphinx
if ($setting['sphinxenable']) {
$sphinx = pc_base::load_app_class('search_interface', '', 0);
$sphinx = new search_interface();
$offset = $pagesize * ($page - 1);
//因所使用的sphinx版本不支持“id”作为排序的名字,所以用news_id替代,并非必须新闻模型
$res = $sphinx->search($q, array($siteid), array($typeid), array($search_time, SYS_TIME), $offset, $pagesize, '@weight desc, news_id desc');
$totalnums = $res['total'];
//如果结果不为空
if (!empty($res['matches'])) {
$result = $res['matches'];
}
}
.......
......
//如果结果不为空
if (!empty($result) || !empty($commend['id'])) {
//开启sphinx后文章id取法不同
if ($setting['sphinxenable']) {
foreach ($result as $_k => $_v) {
//根据sphinx版本不同,$result数组中格式稍有不同,此处稍作修改
// $sids[] = $_v['attrs']['id'];
$sids[] = $_v['attrs']['news_id'];
}
$sort_str = implode(",", $sids);
}
......
......
/**
* 如果表名为空,则为黄页模型
*/
if (empty($this->content_db->model_tablename)) {
$this->content_db = pc_base::load_model('yp_content_model');
$this->content_db->set_model($modelid);
}
if ($setting['sphinxenable']) {
//页面展示的搜索结果没有按照sphinx给出的权重排序,此处进行调整
// $data = $this->content_db->listinfo($where, "id desc", 1, $pagesize);
$data = $this->content_db->listinfo($where, "field(id," . $sort_str . ")", 1, $pagesize);
$pages = pages($totalnums, $page, $pagesize);
} else {
$data = $this->content_db->select($where, '*');
$pages = $this->db->pages;
$totalnums = $this->db->number;
}
......
引入sphinx api
从下载的安装包api目录中找到sphinxapi.php,复制到phpcms>modules>search>classes目录中,并改名为sphinxapi.class.php
定时索引(todo)
配置完毕,如有疑问请留言
参考:sphinx官方文档