MediaCrawler automates Chinese social media scraping

// 1h agoOPENSOURCE RELEASE

MediaCrawler automates Chinese social media scraping

MediaCrawler is an open-source Python framework that uses Playwright-based browser automation to scrape content and comments from major Chinese social media platforms. It simulates authentic user interactions to bypass complex security and platform signing mechanisms natively.

// ANALYSIS

While traditional scraping projects quickly break due to platform security updates and signature changes, MediaCrawler's browser-automation first design makes it exceptionally resilient for scraping heavily protected platforms.

* Playwright integration bypasses complex JS signature and anti-crawling defenses by running logic directly in a real browser context.

* Support for a wide range of popular Chinese social platforms makes it a valuable tool for localized market research, sentiment analysis, and academic study.

* The repository's rapid rise to over 50,000 stars underscores the massive demand for reliable data access to notoriously locked-down Chinese platforms.

* The non-commercial licensing model limits its use in enterprise environments, though the creator offers a Pro version with features like multi-account/proxy handling and API access.

// TAGS

xiaohongshudouyinbilibiliweiboopen-sourceweb-agentautomation

DISCOVERED

1h ago

2026-06-27

PUBLISHED

1h ago

2026-06-27

RELEVANCE

7/ 10

// KEEP READING

More AI developer news from the feed

EXPLORE FULL FEED

OPEN SOURCE45m ago

ACE Robotics, CUHK Open-Source ACE-Ego

ACE ROBOTICS and CUHK have open-sourced ACE-Ego, a unified Vision-Language-Action (VLA) embodied AI model that enables robots to learn from egocentric human videos. The model utilizes camera-space actions and morphology conditioning to translate human movements into robot trajectories, achieving state-of-the-art benchmark performance.

RESEARCH54m ago

BinEval decomposes LLM evaluation into binary questions

BinEval is a training-free, task-agnostic LLM evaluation framework that decomposes complex evaluation criteria into atomic binary questions. By aggregating independent yes/no verdicts, the framework matches or outperforms established baselines like G-Eval while providing interpretable diagnostic feedback for prompt optimization.

OPEN SOURCE1h ago

The popular repository free-for-dev has accumulated over 124,000 stars on GitHub as the premier directory for developer-focused free tiers across SaaS, PaaS, and IaaS offerings.

free-for-dev is a community-curated GitHub repository listing SaaS, PaaS, and IaaS offerings with free tiers of interest to devops and infrastructure developers. Created and maintained by ripienaar, the project helps developers, students, and startups build, test, and host their applications without initial infrastructure costs. It covers a vast range of services including hosting, databases, search, DNS, CI/CD, analytics, and security tools, serving as a comprehensive directory updated daily by the developer community.