Skip to content

python.md

Latest commit

 

History

History
68 lines (49 loc) · 1.82 KB

python.md

File metadata and controls

68 lines (49 loc) · 1.82 KB
Raw

Dev.to Articles Scraper — Python (apify-client) examples

Call the hosted Actor logiover/devto-articles-scraper from Python with the official apify-client.

Install

pip install apify-client

Scrape a tag and read results

from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")

run = client.actor("logiover/devto-articles-scraper").call(run_input={
    "tag": "ai",
    "maxArticles": 200,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item.get("reactionsCount"), "-", item["title"], "- @" + str(item.get("authorUsername")))

Trending analysis into a pandas DataFrame

import pandas as pd
from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN")
run = client.actor("logiover/devto-articles-scraper").call(run_input={
    "sort": "top_week",
    "tag": "webdev",
    "maxArticles": 200,
})

items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
df = pd.DataFrame(items)

# Top 10 by reactions
print(df.sort_values("reactionsCount", ascending=False)[["title", "reactionsCount", "commentsCount"]].head(10))

# Most active authors
print(df["authorUsername"].value_counts().head(10))

Every article from one author

run = client.actor("logiover/devto-articles-scraper").call(run_input={
    "username": "ben",
    "maxArticles": 0,
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["publishedAt"], "-", item["title"])

Export the dataset to CSV

with open("devto.csv", "wb") as f:
    f.write(client.dataset(run["defaultDatasetId"]).download_items(item_format="csv"))

See also: cli.md · api-curl.md · javascript.md