mysql - Searching for existence within 100 million+ strings 5000 times efficiently -


i have text file containing 121 million string, want perform 5000 search existence (i.e. if given string exists in text file or not) , want finish these 5000 checks in 2 seconds or less.

i thought of different ways this, tried put these strings text file sql table primary index on string column , perform query 5000 times:

select * table string=given_string 

then check if result exists or not, way led long execution time whole 5000 query, between 20 , 30 seconds.

i wonder if there efficient way me index/handle data , search among 5000 times in efficient way.

you add (temporary) table (even in memory) , bulk insert 5000 search values in there. afterwards create query joins table big one.

this way have 2 queries instead of 5000.

maybe speeds things bit.


Comments

Popular posts from this blog

sql - invalid in the select list because it is not contained in either an aggregate function -

Angularjs unit testing - ng-disabled not working when adding text to textarea -

How to start daemon on android by adb -