General informations about JotCache extension

TOPIC: Cron: Crawler Extended stops and deletes all found

Cron: Crawler Extended stops and deletes all found 06 Apr 2015 12:32 #1340

Hi

I have edited cron_recache.php with my correct site path and I have commented out the CLI requirement.

The log gives me:

Crawler Extended - depth 1:
2015-04-06 09:23:41 Starting recache run
2015-04-06 09:23:41 .loaded 3 jotcache plugin(s)
2015-04-06 09:23:41 ..registering `crawler` plugin
2015-04-06 09:23:41 ..registering `crawlerext` plugin
2015-04-06 09:23:41 ..registering `recache` plugin
2015-04-06 09:23:41 ...triggering `onJotcacheRecache` event
2015-04-06 09:23:41 ....running in plugin crawlerext
2015-04-06 09:23:46 ....for browser chrome returned 26 links on level 1
2015-04-06 09:24:50 ....during recache with browser:chrome returned 1102 hits
2015-04-06 09:24:50 ...crawlerext plugin returned `STOP`
2015-04-06 09:24:50 Finished recache run

It buids up the URLs i JotCache, but deletes it all afterwards.

Going away from the extended way gives me:

Crawler - depth 1:
2015-04-06 10:17:19 Starting recache run
2015-04-06 10:17:19 .loaded 3 jotcache plugin(s)
2015-04-06 10:17:19 ..registering `crawler` plugin
2015-04-06 10:17:19 ..registering `crawlerext` plugin
2015-04-06 10:17:19 ..registering `recache` plugin
2015-04-06 10:17:19 ...triggering `onJotcacheRecache` event
2015-04-06 10:17:19 ....running in plugin crawler
2015-04-06 10:18:08 ....during recache with browser:chrome returned 1137 hits
2015-04-06 10:18:08 ...crawler plugin returned `DONE`
2015-04-06 10:18:08 Finished recache run

Crawler Extended - depth 5:
2015-04-06 10:20:17 Starting recache run
2015-04-06 10:20:17 .loaded 3 jotcache plugin(s)
2015-04-06 10:20:17 ..registering `crawler` plugin
2015-04-06 10:20:17 ..registering `crawlerext` plugin
2015-04-06 10:20:17 ..registering `recache` plugin
2015-04-06 10:20:17 ...triggering `onJotcacheRecache` event
2015-04-06 10:20:17 ....running in plugin crawler
2015-04-06 10:21:49 ....during recache with browser:chrome returned 1994 hits
2015-04-06 10:21:49 ...crawler plugin returned `DONE`
2015-04-06 10:21:49 Finished recache run

In other words the extended crawling is not working. It stops and deletes it found URLs in one go.

So for now I will just use non-extended way because it is working.

I have set the php timeout to 600 seconds (we have our own dedicated server). It is not the problem. It is not giving a timeout.

I am using the Release Candiate of JotCache.
The administrator has disabled public write access.

Re: Cron: Crawler Extended stops and deletes all found 10 Apr 2015 09:00 #1345

JotCache crawlers never delete cached pages. It must be some other external (manual) process.
The administrator has disabled public write access.
Time to create page: 0.289 seconds
We have 82 guests and no members online
Copyright © 2015 JotComponents
We have 82 guests and no members online
Copyright © 2017 JotComponents