BeautifulSoup getting href

PythonTagsBeautifulsoup

Python Problem Overview


I have the following soup:

<a href="some_url">next</a>
<span class="class">...</span>

From this I want to extract the href, "some_url"

I can do it if I only have one tag, but here there are two tags. I can also get the text 'next' but that's not what I want.

Also, is there a good description of the API somewhere with examples. I'm using the standard documentation, but I'm looking for something a little more organized.

Python Solutions


Solution 1 - Python

You can use find_all in the following way to find every a element that has an href attribute, and print each one:

from BeautifulSoup import BeautifulSoup

html = '''<a href="some_url">next</a>
<span class="class"><a href="another_url">later</a></span>'''

soup = BeautifulSoup(html)

for a in soup.find_all('a', href=True):
    print "Found the URL:", a['href']

The output would be:

Found the URL: some_url
Found the URL: another_url

Note that if you're using an older version of BeautifulSoup (before version 4) the name of this method is findAll. In version 4, BeautifulSoup's method names were changed to be PEP 8 compliant, so you should use find_all instead.


If you want all tags with an href, you can omit the name parameter:

href_tags = soup.find_all(href=True)

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestiondkgirlView Question on Stackoverflow
Solution 1 - PythonMark LongairView Answer on Stackoverflow