AbotX : How do you create a parallel crawler that stays on and can be added to at run time from new requests -

-

i have parallelcrawlerengine setup singleton , have alwaysonsitetocrawlprovider set singleton , passed parallelcrawlerengine.

i can instantiate , leave nothing ok. can add site , crawl ok. if add site not crawl second site.

i have looked @ example on site doesn't appear show how work , have new items added after initial execution. using .addsitestocrawl() adds them list seems stay in purgatory state of not being read.

looking through logs 'site completed' message though site has not been recrawled 

[2016-07-11 11:17:18,361] [20 ] [info ] - crawl domain [http://www.existingsite.com/] completed in [0.0001118] seconds, crawled [361] pages

and error if add new site

[2016-07-11 11:17:33,365] [23 ] [error] - crawl domain [http://www.newsite.com/] failed after [0.0066498] seconds, crawled [361] pages [2016-07-11 11:17:33,365] [23 ] [error] - system.invalidoperationexception: cannot call dowork() after abortall() or dispose() have been called.    @ abot.util.threadmanager.dowork(action action)    @ abot.crawler.webcrawler.crawlsite()    @ abot.crawler.webcrawler.crawl(uri uri, cancellationtokensource cancellationtokensource)


Comments

  1. Apoorv Chaudhary23 February 2022 at 04:20

    The article has nicely shown the skills of this java programming software tool, SonarQube With extensive OWASP Top 10 insurance for Java, SonarQube brings troubles to developers early in the method to help you defend your structures, your information, and your users. Push notifications and open APIs make integrating with your different structures painless.

    ReplyDelete

Post a Comment

Popular posts from this blog

sql - invalid in the select list because it is not contained in either an aggregate function -

-
i can't figure out rewrite sql statement avoid error below msg 8120, level 16, state 1, line 26 column 'staffmember.staffmemberid' invalid in select list because not contained in either aggregate function or group clause. sql statment select (select [dbo].[udf_readstaffmemberbasic](sm.staffmemberid) xml path ('staffmembersummary'), root ('items'), type ), count(sm.staffmemberid) totalresults, ceiling(cast(count(sm.staffmemberid) decimal)/ @pagesize) totalpages staffmember sm inner join staffmembertoadditionalrole ar on sm.staffmemberid = ar.staffmemberid (sm.firstname '%' + @name + '%' or sm.surname '%' + @name + '%') , (@floorid null or sm.floorid = @floorid) , (@directorateid null or sm.directorateid = @directorateid) , (@additionalroleid null or ar.additionalroleid = @additionalroleid) order sm.staffmemberid offset @page * @pagesize rows fe...
Read more

Angularjs unit testing - ng-disabled not working when adding text to textarea -

-
i trying test angularjs component using karma, mocha, power-assert. have textarea , button in component, button disabled based on length of text in textarea. when run component in browser works perfectly. can't figure out how test functionality. here's bit of code. inquiryform.js function inquiryform($scope) { // plan add logic here } angular.module('myapp').component('inquiryform', { controller: inquiryform, controlleras: 'inquiryform', templateurl: 'inquiryform.html' }); inquiryform.html <div class="contact"> <div>thanks contacting us.</div> <form> <textarea ng-model="inquiryform.inquirytext" name=""></textarea> <input type="submit" value="send" ng-disabled="!inquiryform.inquirytext.length"> </form> </div> inquiryformtest.js describe('inquiry form ', () => { let $r...
Read more

How to start daemon on android by adb -

-
can me please, head swollen... i trying run daemon on android emu/device command: adb -s <device_name> shell su -c /dir/daemon <port_number> but nothink happens, , no errors! if do: adb -s <device> shell and form shell cmdline: su -c /dir/daemon <port_number> than work fine. try use shell-script , run: adb -s <device_name> shell sh su -c /dir/script.sh <port_number> and try generate script qt code port number , use: adb -s <device_name> shell sh su -c /dir/script.sh but not helped... problem if enter adb shell previous run daemon - work. problem on different device/emu/iso_image different command format not work. example: adb -s <device_name> shell su -c /dir/daemon <port_number> // work @ emu, not @ iso and vice: adb -s <device_name> shell su -c "/dir/daemon <port_number>" // work @ iso etc. everybody can answer what's matter? sorry english adb shell su -c ...
Read more