Minimalist wb formats: WAP2...

Periodically, I've been web-scraping my own arduino posts, in case (say) the forum software gets upgraded to something new and incompatible and older posts are lost.

I use a little C program that generates a bunch of commands for wget, using URLs like:

https://forum.arduino.cc/index.php?action=profile;u=2336;area=showposts;wap2;start=100

This USED to return a sort of minimalist form of the arduino forum pages, without all the extra headers and buttons and stuff. At the moment, it seems to just open the same busy page as without the WAP2 qualifier.

I know the WAP2 pages existing used to be pain for search engines, but... is there some replacement minimalist display format that I can use?

It looks like everything now goes through Cloudflare. It's possible those of us on the internet side can no longer even reach the actual forum server. My suspicion is the "wap2" is stripped as the request passes through Cloudflare. If all that's true, what you want is no longer available.

It's my understanding Beautiful Soap (Python) is to go-to solution for web scraping.

Sorry I don't have better news.

It's my understanding Beautiful Soap (Python) is to go-to solution for web scraping.

I've used it, and it's pretty amazing. I've got half-a-page of code that did a pretty good job of scraping a web-comic I like. But the WAP2 format got me almost all I wanted without having to do any analysis of the forum internal format...