Get Info From Script Tag (WebScrape) [duplicate]

Question

#Python Code
from bs4 import BeautifulSoup
import urllib3

url ='https://www. SomeData .com'
req = urllib3.PoolManager()
res = req.request('GET', url)
soup = BeautifulSoup(res.data, 'html.parser')
res = soup.find_all('script')
print(res)

Then I Got something like this:

Results below:
[
  <script>
        AAA.trackData.taxonomy = {
              a:"a",
              b:"b",
              c:"c2,
              ...} ;
</script>
</script>, <script async="" src="https://someData.com/js/detail.0a6eca28.js"></script>
]

How can i convert this to a json format to treat well data inside tag.

Whats your desired output?

heemayl
– heemayl

2019-07-27 12:16:57 +00:00
Commented Jul 27, 2019 at 12:16 — heemayl
– heemayl, Commented Jul 27, 2019 at 12:16

LazyCoder · Accepted Answer · 2019-07-27 12:26:25Z

0

Please check if this helps.

script = soup.find('script', text=re.compile('AAA\.trackData\.taxonomy'))
json_text = re.search(r'^\s*AAA\.trackData\.taxonomy\s*=\s*({.*?})\s*;\s*$',
                      script.string, flags=re.DOTALL | re.MULTILINE).group(1)
data = json.loads(json_text)

answered Jul 27, 2019 at 12:26

LazyCoder

1,2637 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Get Info From Script Tag (WebScrape) [duplicate]

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related