php call scrapy by custom command

Question

I have to run my five different scrapy spiders at the same time so I define a custom command in scrapy like

scrapy crawlall

to call these spiders. It works successfully when I run in command line in independent.

However it fails when I use php shell_exec to call the same command and the code in php file is like

$cmd = 'scrapy crawlall';
$results = shell_exec($cmd);
echo $results;

The web page will echo nothing immediately and not waiting the scrapy function(or even it wasn't be called at the beginning I don't know). So how can I call this command or other way to run these 5 spiders at the same time in php file? Thanks!

I found that the command was called and excute but stop by it self at: [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023 Does anyone know what's going on? — 許家瑄
– 許家瑄, Commented Jan 24, 2019 at 4:45
Are you sure it is working until then? Can you try running scrapy list from PHP and see if it returns the expected spiders? — Gallaecio
– Gallaecio, Commented Jan 29, 2019 at 17:17

許家瑄 · Accepted Answer · 2019-01-30 02:02:18Z

0

I have solved the problem. If you get the same situation, try to remove all comment in your command python file. I don't know the reason but after I do that it works successful.

answered Jan 30, 2019 at 2:02

許家瑄

974 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

php call scrapy by custom command

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related