Import Hosting Handbook Markdown from Github

[kirasong]

In #5240, a hosting handbook repo was created: ​https://github.com/wordpress/hosting-handbook

Now, I'm looking to figure out the best way to set up automated import of the Markdown there.

I noticed that both WP-CLI and Gutenberg are set up to auto-import, with slightly different ways of using the library plugin.

I've tested importing on a VVV meta repo, but am having trouble testing the styles.

Attached is a first draft of the WP-CLI handbook plugin ported to the hosting handbook. Not sure if it's better to go this route, or duplicate less code somehow.

[kirasong]

First pass -- Hosting Handbook importer plugin.

[kirasong]

Ah, I should note, I did notice that assets/images don't seem to get added to the WP media library on import.

I'm not sure if I did something wrong there, if they should be transformed, or if the intended behavior is to link to the assets elsewhere.

[dd32]

Replying to mikeschroder:

Ah, I should note, I did notice that assets/images don't seem to get added to the WP media library on import.

I don't think there's any handling for images, and I'm not sure any of the existing handbooks that use GitHub have them.

[kirasong]

Replying to dd32:

I don't think there's any handling for images, and I'm not sure any of the existing handbooks that use GitHub have them.

Thanks; I wondered if maybe that was why Gutenberg's docs ​seem to use Cloudup for images. The hosting handbook only has two images at the moment, so it wouldn't be hard to move them wherever they need to be, then document how to add future images.

[dd32]

so it wouldn't be hard to move them wherever they need to be, then document how to add future images.

I would recommend just uploading them to the media library manually, and inserting the wordpress.org link into the markdown :)
If that becomes a problem long-term, someone can offer to add sideloading functionality to the handbook plugins.

[kirasong]

Replying to dd32:

I would recommend just uploading them to the media library manually, and inserting the wordpress.org link into the markdown :)
If that becomes a problem long-term, someone can offer to add sideloading functionality to the handbook plugins.

Thanks! Sounds like a plan.

These are already uploaded, and the links have been changed + docs added.

[dd32]

@coffee2code Is there a new importer that doesn't need to be copy-pasted every time? I think you're far more familiar than me with the handbooks..

[coffee2code]

Replying to dd32:

@coffee2code Is there a new importer that doesn't need to be copy-pasted every time? I think you're far more familiar than me with the handbooks..

Unfortunately, no. I believe heretofore, no other site other than DevHub had use for a GitHub docs importer until now. The Gutenberg demo site did have its own version introduced a couple months after DevHub introduced a specialized one for the REST API docs. But that was decommissioned early last year as the DevHub docs importer was abstracted into a base class to accommodate multiple imported docs (REST API, Coding Standards, and Block Editor). At which point the Gutenberg site docs importer was itself decommissioned.

The base docs importer code is only minimally DevHub-branded and specific and could probably be made general-purpose and integrated into the Handbooks plugin. I'll give it a go.

[kirasong]

Replying to coffee2code:

Unfortunately, no. I believe heretofore, no other site other than DevHub had use for a GitHub docs importer until now.

Thanks so much for looking into this! It would be great to have something more abstracted/general.

I'm pretty sure you're aware, but just to be sure, it looks like the WP-CLI Handbook / plugin ​also uses a Markdown importer, and that's the one I initially based off of.

[coffee2code]

@mikeschroder: Just to confirm, has all of the data in the existing handbook pages (on make/hosting) been incorporated into the GH version? I don't want to inadvertently overwrite anything that'll be missed.

[kirasong]

Replying to coffee2code:

@mikeschroder: Just to confirm, has all of the data in the existing handbook pages (on make/hosting) been incorporated into the GH version? I don't want to inadvertently overwrite anything that'll be missed.

Yes! It was all ported to the Github version, and it has been worked on there since.

[kirasong]

Hey, just thought I'd check in here -- is there anything you need / any way I can help out?

[kirasong]

@coffee2code Thought I'd check in on this again. There have been some parts waiting to be updated/published since this request was created here.

Do you know what the timeframe looks like? Wondering if we should update them manually, or in some other fashion.

Also happy to help / put together another pass if it is helpful, but wasn't sure where you were on the process.

Thanks again!

[coffee2code]

The Handbooks plugin has just recently been updated to facilitate handling importing of handbooks from GH for Make sites, so this should just about be ready to go.

I just need someone(s) from the team to confirm if all existing content in the ​Make/Hosting handbook is contained in the GitHub version of the handbook. Those pages will ultimately get overwritten during an import, so I want to make sure we won't lose anything.

I'll also have to take a closer look at the ​manifest.json file, as that's the important bit for directing the import. Already I can see it's missing a reference to the Tests page. There are probably a few other tweaks as well that I can submit a PR for. Tnen I'll do a local test import to ensure the doc pages look right.

We can make this happen next week.

[JavierCasares]

Thank you!

I just created a PR with the updated manifest

​https://github.com/WordPress/hosting-handbook/pull/79

I hope today will be approved and merged :)

[coffee2code]

I see ​PR79 got recently merged. I just filed ​PR80 to make a few changes:

A few things that should be addressed prior to initial import:

• Reorder manifest entries to match order currently found in live handbook. (To retain order of pages.)
• Don't escape forward slashes in manifest. (While not problematic, it's not necessary.)
• Don't escape brackets for shortcodes. (Escaping prevents them from being treated as shortcodes.)

I believe after that gets merged the handbook can be imported.

However, I'd like restate my request for confirmation that all content in the ​existing Hosting Handbook has been incorporated into the handbook as found on GitHub. The import will overwrite whatever is there.

[coffee2code]

I just noticed that all pages of the handbook are child pages of the handbook landing page. This is not advised as it doesn't really make sense here and leads to having a URL like https://make.wordpress.org/hosting/handbook/handbook/performance/ (having "handbook" in the path twice).

Fortunately the manifest does not reflect this unnecessary hierarchy. Before the import, I'll have to unparent all the pages and create redirects to ensure preexisting links continue to work.

[coffee2code]

Since my ​PR80 was recently merged, I just want to get two confirmations from @JavierCasares or another team rep before proceeding with the handbook import:

1. Confirmation that all content in the production Hosting Handbook has been merged into the GitHub version.
   Per comment 20:

   I just need someone(s) from the team to confirm if all existing content in the ​Make/Hosting handbook is contained in the GitHub version of the handbook. Those pages will ultimately get overwritten during an import, so I want to make sure we won't lose anything.
   

2. Confirmation that the un-parenting of handbook pages from the root page is fine, per my comment in comment 25. (All pages will become root pages.)

Thanks!

[JavierCasares]

Replying to coffee2code:

Since my ​PR80 was recently merged, I just want to get two confirmations from @JavierCasares or another team rep before proceeding with the handbook import:

I told the Reps to check it :)

1. Confirmation that all content in the production Hosting Handbook has been merged into the GitHub version.
   Per comment 20:

   I just need someone(s) from the team to confirm if all existing content in the ​Make/Hosting handbook is contained in the GitHub version of the handbook. Those pages will ultimately get overwritten during an import, so I want to make sure we won't lose anything.
   

OK for me. Some time ago we started to update the Git and then, copying manually from the Git to the Handbook, so, it should be right :)

2. Confirmation that the un-parenting of handbook pages from the root page is fine, per my comment in comment 25. (All pages will become root pages.)

OK for me, I never understand why it was like it was.

[Crixu]

Replying to coffee2code:

Since my ​PR80 was recently merged, I just want to get two confirmations from @JavierCasares or another team rep before proceeding with the handbook import:

Both changes are ok for me

[coffee2code]

Implemented.

In [17479-dotorg]:

Make/Hosting: Add redirects for pages that used to be relative to the root handbook page but are all now root handbook pages themselves.

As noted above, all but the handbook root/landing page were sub-pages of the handbook root/landing page. Now all pages are root pages. This redirects the old URLs.

In [17480-dotorg]:

Make/Hosting: Explicitly configure handbook so that it can now be imported from GitHub.

Props mikeschroder, javiercasares, crixu, coffee2code.
Fixes #meta5247.

Everything seems to have imported properly. Let me know if there is anything is amiss.