Extract specific value from HTML using scrapy and css

Question

I have the following HTML code:

<div class="sites-split">
    <ul>
        <li><a href="http://www.page1.com/" target="_blank">text 1</a></li>
        <li><a href="http://www.page2.com/" target="_blank">text 2</a></li>
    </ul>
    <ul>
        <li><a href="http://www.page3.com/" target="_blank">text 3</a></li>
        <li><a href="http://www.page4.com/" target="_blank">text 4</a></li>
    </ul>
</div>

I want to extract te links and the text associated to that link, I manage to get the links by doing:

response.css("div.sites-split a::attr(href)").getall()

But I don't know how to get the text, I tried:

response.css("div.sites-split a::attr(target)").getall()

But all I get is:

['_blank',
 '_blank',
 '_blank',
 '_blank']

vezunchik · Accepted Answer · 2019-02-13 15:24:06Z

4

You're extracting another attribute, but you need text. Try response.css("div.sites-split a::text").getall()

answered Feb 13, 2019 at 15:24

vezunchik

3,7173 gold badges20 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Extract specific value from HTML using scrapy and css

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related