The Scraping Wiki: An LLM-maintained knowledge base indexing 400 articles

[Skip to content](https://github.com/TheWebScrapingClub/scraping-wiki/blob/main/index.md#start-of-content) ## Navigation Menu Toggle navigation [ ](https://github.com/) [ Sign in ](https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FTheWebScrapingClub%2Fscraping-wiki%2Fblob%2Fmain%2Findex.md) Appearance settings * Platform * AI CODE CREATION * [ GitHub CopilotWrite better code with AI ](https://github.com/features/copilot) * [ GitHub SparkBuild and deploy intelligent apps ](https://github.com/features/spark) * [ GitHub ModelsManage and compare prompts ](https://github.com/features/models) * [ MCP RegistryNewIntegrate external tools ](https://github.com/mcp) * DEVELOPER WORKFLOWS * [ ActionsAutomate any workflow ](https://github.com/features/actions) * [ CodespacesInstant dev environments ](https://github.com/features/codespaces) * [ IssuesPlan and track work ](https://github.com/features/issues) * [ Code ReviewManage code changes ](https://github.com/features/code-review) * APPLICATION SECURITY * [ GitHub Advanced SecurityFind and fix vulnerabilities ](https://github.com/security/advanced-security) * [ Code securitySecure your code as you build ](https://github.com/security/advanced-security/code-security) * [ Secret protectionStop leaks before they start ](https://github.com/security/advanced-security/secret-protection) * EXPLORE * [Why GitHub](https://github.com/why-github) * [Documentation](https://docs.github.com) * [Blog](https://github.blog) * [Changelog](https://github.blog/changelog) * [Marketplace](https://github.com/marketplace) [View all features](https://github.com/features) * Solutions * BY COMPANY SIZE * [Enterprises](https://github.com/enterprise) * [Small and medium teams](https://github.com/team) * [Startups](https://github.com/enterprise/startups) * [Nonprofits](https://github.com/solutions/industry/nonprofits) * BY USE CASE * [App Modernization](https://github.com/solutions/use-case/app-modernization) * [DevSecOps](https://github.com/solutions/use-case/devsecops) * [DevOps](https://github.com/solutions/use-case/devops) * [CI/CD](https://github.com/solutions/use-case/ci-cd) * [View all use cases](https://github.com/solutions/use-case) * BY INDUSTRY * [Healthcare](https://github.com/solutions/industry/healthcare) * [Financial services](https://github.com/solutions/industry/financial-services) * [Manufacturing](https://github.com/solutions/industry/manufacturing) * [Government](https://github.com/solutions/industry/government) * [View all industries](https://github.com/solutions/industry) [View all solutions](https://github.com/solutions) * Resources * EXPLORE BY TOPIC * [AI](https://github.com/resources/articles?topic=ai) * [Software Development](https://github.com/resources/articles?topic=software-development) * [DevOps](https://github.com/resources/articles?topic=devops) * [Security](https://github.com/resources/articles?topic=security) * [View all topics](https://github.com/resources/articles) * EXPLORE BY TYPE * [Customer stories](https://github.com/customer-stories) * [Events & webinars](https://github.com/resources/events) * [Ebooks & reports](https://github.com/resources/whitepapers) * [Business insights](https://github.com/solutions/executive-insights) * [GitHub Skills](https://skills.github.com) * SUPPORT & SERVICES * [Documentation](https://docs.github.com) * [Customer support](https://support.github.com) * [Community forum](https://github.com/orgs/community/discussions) * [Trust center](https://github.com/trust-center) * [Partners](https://github.com/partners) [View all resources](https://github.com/resources) * Open Source * COMMUNITY * [ GitHub SponsorsFund open source developers ](https://github.com/sponsors) * PROGRAMS * [Security Lab](https://securitylab.github.com) * [Maintainer Community](https://maintainers.github.com) * [Accelerator](https://github.com/accelerator) * [GitHub Stars](https://stars.github.com) * [Archive Program](https://archiveprogram.github.com) * REPOSITORIES * [Topics](https://github.com/topics) * [Trending](https://github.com/trending) * [Collections](https://github.com/collections) * Enterprise * ENTERPRISE SOLUTIONS * [ Enterprise platformAI-powered developer platform ](https://github.com/enterprise) * AVAILABLE ADD-ONS * [ GitHub Advanced SecurityEnterprise-grade security features ](https://github.com/security/advanced-security) * [ Copilot for BusinessEnterprise-grade AI features ](https://github.com/features/copilot/copilot-business) * [ Premium SupportEnterprise-grade 24/7 support ](https://github.com/premium-support) * [Pricing](https://github.com/pricing) Search or jump to... # Search code, repositories, users, issues, pull requests... Search Clear [Search syntax tips](https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax) # Provide feedback We read every piece of feedback, and take your input very seriously. Include my email address so I can be contacted Cancel Submit feedback # Saved searches ## Use saved searches to filter your results more quickly Name Query To see all available qualifiers, see our [documentation](https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax). Cancel Create saved search [ Sign in ](https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FTheWebScrapingClub%2Fscraping-wiki%2Fblob%2Fmain%2Findex.md) #### Sign in to GitHub Username or email address Password [Forgot password?](https://github.com/password_reset) [ or continue with other methods ](https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FTheWebScrapingClub%2Fscraping-wiki%2Fblob%2Fmain%2Findex.md) [ Sign up ](https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2FTheWebScrapingClub%2Fscraping-wiki%2Fblob%2Fmain%2Findex.md&source=header) Appearance settings Resetting focus You signed in with another tab or window. [Reload](https://github.com/TheWebScrapingClub/scraping-wiki/blob/main/index.md) to refresh your session. You signed out in another tab or window. [Reload](https://github.com/TheWebScrapingClub/scraping-wiki/blob/main/index.md) to refresh your session. You switched accounts on another tab or window. [Reload](https://github.com/TheWebScrapingClub/scraping-wiki/blob/main/index.md) to refresh your session. Dismiss alert {{ message }}         Find code, projects, and people on GitHub: Search [Contact Support](https://support.github.com?tags=dotcom-404) — [GitHub Status](https://githubstatus.com) — [@githubstatus](https://x.com/githubstatus) ## Site-wide Links [ ](https://github.com/) ### Subscribe to our developer newsletter Get tips, technical guides, and best practices. Twice a month. [ Subscribe ](https://github.com/newsletter) ### Platform * [Features](https://github.com/features) * [Enterprise](https://github.com/enterprise) * [Copilot](https://github.com/features/copilot) * [AI](https://github.com/features/ai) * [Security](https://github.com/security) * [Pricing](https://github.com/pricing) * [Team](https://github.com/team) * [Resources](https://resources.github.com) * [Roadmap](https://github.com/github/roadmap) * [Compare GitHub](https://github.com/resources/articles/devops-tools-comparison) ### Ecosystem * [Developer API](https://docs.github.com/get-started/exploring-integrations/about-building-integrations) * [Partners](https://partner.github.com) * [Education](https://github.com/edu) * [GitHub CLI](https://cli.github.com) * [GitHub Desktop](https://desktop.github.com) * [GitHub Mobile](https://github.com/mobile) * [GitHub Marketplace](https://github.com/marketplace) * [MCP Registry](https://github.com/mcp) ### Support * [Docs](https://docs.github.com) * [Community Forum](https://github.community) * [Professional Services](https://services.github.com) * [Premium Support](https://github.com/enterprise/premium-support) * [Skills](https://skills.github.com) * [Status](https://www.githubstatus.com) * [Contact GitHub](https://support.github.com?tags=dotcom-footer) ### Company * [About](https://github.com/about) * [Why GitHub](https://github.com/why-github) * [Customer stories](https://github.com/customer-stories?type=enterprise) * [Blog](https://github.blog) * [The ReadME Project](https://github.com/readme) * [Careers](https://github.careers) * [Newsroom](https://github.com/newsroom) * [Inclusion](https://github.com/about/diversity) * [Social Impact](https://socialimpact.github.com) * [Shop](https://shop.github.com) * © 2026 GitHub, Inc. * [Terms](https://docs.github.com/site-policy/github-terms/github-terms-of-service) * [Privacy](https://docs.github.com/site-policy/privacy-policies/github-privacy-statement) [(Updated 02/2024)02/2024](https://github.com/github/site-policy/pull/582) * [Sitemap](https://github.com/sitemap) * [What is Git?](https://github.com/git-guides) * Manage cookies * Do not share my personal information * [ GitHub on LinkedIn ](https://www.linkedin.com/company/github) * [ Instagram GitHub on Instagram ](https://www.instagram.com/github) * [ GitHub on YouTube ](https://www.youtube.com/github) * [ GitHub on X ](https://x.com/github) * [ TikTok GitHub on TikTok ](https://www.tiktok.com/@github) * [ Twitch GitHub on Twitch ](https://www.twitch.tv/github) * [ GitHub’s organization on GitHub ](https://github.com/github) English * [ English ](https://github.com/TheWebScrapingClub/scraping-wiki/blob/main/index.md) * [ Português (Brasil) ](https://github.com/TheWebScrapingClub/scraping-wiki/blob/main/index.md) * [ Español (América Latina) ](https://github.com/TheWebScrapingClub/scraping-wiki/blob/main/index.md) * [ 日本語 ](https://github.com/TheWebScrapingClub/scraping-wiki/blob/main/index.md) * [ 한국어 ](https://github.com/TheWebScrapingClub/scraping-wiki/blob/main/index.md)