Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

能否加入并行分词功能? #53

Open
bourdieu opened this issue May 2, 2017 · 1 comment
Open

能否加入并行分词功能? #53

bourdieu opened this issue May 2, 2017 · 1 comment

Comments

@bourdieu
Copy link

bourdieu commented May 2, 2017

如果使用并行包,则会总是提醒需要添加worker
如果把新建worker的命令放在一个自编函数里再并行运行,由于每一次都需要重新初始化worker所以就十分慢
能否如同Python版的jieba,增加对并行分词的支持呢?
祝好!

@qinwf
Copy link
Owner

qinwf commented May 2, 2017

您好,新建一个 worker ,这个 worker 以后可以重复使用很多次,不需要重新新建也能分词。修改参数可以使用 workername$参数修改。

并行是可以支持的,只需要在各自的进程里新建 worker 就可以了,而不是所有进程共用一个 worker 。我之后写一个示例。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants