Skip to content

Commit

Permalink
turn SQLite journaling off, for performance (gh-11)
Browse files Browse the repository at this point in the history
Indexing the first 10000 articles from an nlwiki dump previously took:

real    1m17.655s
user    0m48.409s
sys     0m25.430s

Now:

real    0m46.042s
user    0m40.603s
sys     0m4.408s
  • Loading branch information
larsmans committed Nov 24, 2014
1 parent 47a68a7 commit 8ed7a84
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions semanticizest/parse_wikidump/createtables.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
pragma foreign_keys = on;
pragma journal_mode = off;

drop table if exists linkstats;
drop table if exists ngrams;
Expand Down

1 comment on commit 8ed7a84

@larsmans
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Performance on the whole nlwiki dump: 50mins, down from 1h15.

Please sign in to comment.