Project Browser Initiative Meeting for Feb 9, 2022 [#3263704]

1️⃣ Issue #3224709: [API] Include API translation layer with mock/fixture implementation until d.o API is ready - I would like to look into the code and come up with some concrete steps someone can follow to get working on this. At the synchronous call, we decided to replace the database fixture with an API fixture with example data. That is, we need to write a submodule that is a (and for now the default) backend API plugin used by the system. We can then manipulate this "fake API" until we've settled on the schema that we want. @nod_ suggests doing this in a submodule, which I totally agree with. How would that module "plug in" to the backend API as it currently stands?

chrisfromredfin	Example TBD: from where should we read the fake data? SQLite? a bigole' JSON file? Should we invent data using Faker?
Rajab Natshah	I'm with SQLite, which feeds a custom database connection to the project_browser_api module.Configs of the Drupal Core JSON:API would be placed in the module to read from the SQLite small db and present the API to the project_browser module as an internal proxy (edited)
mradcliffe	Advantage of JSON storage is it could have a unminified form which is easier to read in patches / diffs more so than a binary database file.
fjgarlin	perhaps having a connector interface to establish the method signatures, and then submodules (or external modules) can be created to plugin to json files, csv, google sheets, json-endpoint… and eventually d.org . each (sub)module should take care of reading the data from the right place. (edited)
chrisfromredfin	@fjgarlin I do agree that's overall the architecture we want; I guess the question here is - for our mock API, which of those choices is best.
fjgarlin	I’ve seen a few google sheets flying around in previous meetings, so perhaps that’s a good starting point? but if that’s too complex any self hosted file in the module could also do for now
chrisfromredfin	https://git.drupalcode.org/project/project_browser/-/blob/1.0.x/src/Proj...
chrisfromredfin	The idea is anything that implements the ProjectBrowserEndpointInterface can be plugged in (tho I'm not sure how specifically it does that). I'm thinking they're implemented as plugins, but haven't looked.
chrisfromredfin	Yeah, the sheets are mostly around data we've been trying to collect to help the Site Builder Subcommittee get to an understanding of what are our "top" modules. I would think we could initially populate JSON file(s) from some sort of scripted process that scrapes what real data we can from the API, and then fakes the data we don't yet have. ?
chrisfromredfin	(or similar for SQLite)
chrisfromredfin	we could ship either with the module, and provide a script / hook / method to replace the fixture... ?
Rajab Natshah	Yes, anything that implement a normal Drupal Database connection and query for search to feed the API endpoint
mradcliffe	What's the format for the description going to be? Many of the modules have an initial h2 heading in their short description.
mradcliffe	Or html lists
chrisfromredfin	The hope is that there's no heading there and it gets replaced with a two-sentence intro describing why I might use that module.
chrisfromredfin	We have a template @AmyJune (volkswagenchick) came up with - #3230734: Improve project descriptions by using a template suggestion for the body field - and the idea is that the SBS is first identifying our top 100 modules and then will be submitting issues to module maintainers to re-work their pages in a way compatible with the template (and therefore easy for us to scrape the first 60 characters or whatever in a meaningful way)
AmyJune (volkswagenchick)	Thanks Chris for mentioning this.
chrisfromredfin	see thread 4️⃣ for more info; I think in fact that the current working sheet has descriptions marked as "good" but that might not have been with these important ideas ^^ in mind; so that may need some re-work. Right @leslieg?
leslieg	Yes, that’s correct. @AmyJune (volkswagenchick) Once we finish the spreadsheet maybe we have Discover Drupal students give feedback on the short descriptions
chrisfromredfin	OK I've looked through and I don't think there's a plugin system yet implemented for sources. That's definitely something that could be worked on. But do we think the right thing for now is to just replace this with calls to our mock API? https://git.drupalcode.org/project/project_browser/-/blob/1.0.x/src/Cont...
chrisfromredfin	Any consensus on JSON v SQLite? I feel like SQLite requires changes to settings.php to set it up as an additional database... maybe JSON will be easier. Just read it and go. ?
chrisfromredfin	Could use something like jsonq (https://github.com/nahid/jsonq) to interact with the fixture/store / respond to queries, etc.
fjgarlin	Agree on json. Easier to get the first iteration and less dependencies
nod_	Not sure I understand why this needs to be pluggable? In my mind this api submodule code is a throwaway, the result of the whole thing is the API contract. This can be horrible, hardcoded, non performant code, it is not destined to be added to Drupal core. Helping the DA with implementation is nice but it shouldn't hold up the work on the submodule for the purpose of demo. As for the JSON/sqlite discussion not sure what is the purpose, we already have the module data thanks to the fixture? The API implementation can definitely reuse those tables to do it's thing for now.
Rajab Natshah	Ahaaa, totally with @nod_Custom table in the Drupal default database ( which it's the fixture for now ) Integration with the JSON:API module to read data from provided tables.Design the needed fake Project Browser endpoints from that.Feed the fake endpoints as a local-proxy for example instead of the drupal-org-proxyWhen the sub module is enabled ( It feels like a plugin to the Project Browser, and a temp local Project Browser Store too )Thanks @nod_ my head went to something else (edited)
chrisfromredfin	not saying this fixture needs to be pluggable, saying that Project Browser should have a way to plug in different backends, whether they be Drupal.org, someone else's custom endpoint, or a mock.
chrisfromredfin	But yeah we could use the existing fixture tables, works better than either. Was just wondering if we wanted to do a smaller subset of data or something easier to finagle. I don't know how the existing fixture setup works, which is part of my ignorance here.
nod_	ah yes, need to be somewhat pluggable for the module to have custom endpoints. It's not going to be in the api submodule that happens so it's a separate topic that should be tracked independently
nod_	@Rajab Natshah I would skip the json:api step in your list. The implementation on d.o probably won't be using it (performance/caching, reliability of updating drupal versions, etc.) and doing quick & dirty hardcoded endpoint will be probably faster and work just as well
chrisfromredfin	^ agree here, no need for JSON:API, just a "json api" perhaps. And yes, as far as issues go, I would think separate. Just having the discussion here to figure out which issues need to get opened/updated/filed.

2️⃣ Until we get the work in #1 done, it may be time to yet again update the existing DB fixture so we're semi-functional. See #3259657: Update project fixture ?

Gábor Hojtsy (he/him)	I don’t know if Rajab’s chechmark means it will be done by Rajab, but in case not, I can take this up for a spin.
chrisfromredfin	No his checkmark usually means "I've read this one" :slightly_smiling_face:
chrisfromredfin	would certainly appreciate it, @Gábor Hojtsy (he/him) :slightly_smiling_face: (edited)
Rajab Natshah	Only suggesting a custom Drush command to update the fixture locally. not to keep doing this in code.or a Drush command to build a local small db or sheet or csv file for the other way.
Rajab Natshah	Testing will be in 3 stepsRequire the moduleFetch PB fixture or dataInstall the module using the development mode
chrisfromredfin	yaaas custom drush command. exactly what I was thinking except, I dunno, forgot about Drush or something. But yes, on-demand only. A tool for devs to update it kinda like we have an update hook now for updating the DB fixture (and it takes a few hours :wink: )
Rajab Natshah	very well. A a custom bash script command will do for sureAdding a limit for the fetcher is important too
shaal	@chrisfromredfin I just spinned it from the module page, and it's not usable No records available
chrisfromredfin	ok let's hope @Gábor Hojtsy (he/him) updating the fixture helps again.
Gábor Hojtsy (he/him)	Yeah it probably will help

3️⃣ I would like to continue the debate of Svelte vs. Vanilla Javascript, and see if we can come to consensus for our short term and long term approaches. I am starting an issue for this now, I would like to have the discussions there so we can pro & con it, keeping an updated summary going, until we feel like we can make a decision.

chrisfromredfin	I slapped a few main points on this issue to kick us off: #3263443: Consider removing SvelteJS in favor of vanilla JavaScript
leslieg	Wasn’t there an issue of not being able to get this in core if we use SvelteJS? @chrisfromredfin
chrisfromredfin	Yes I mentioned that in the first bullet for Vanilla
chrisfromredfin	feel free to edit issue summary if not prominent enough
leslieg	sorry, read that but missed it. It seems like a major blocker to me, but I’ll leave discussion to the more technical folks

5️⃣ I would also like to start planning for DrupalCon. Leslie and I are planning to be there, and I plan to use a good chunk of my time in the sprint room actively working on Project Browser. Should we try and coordinate something more formal? Any ideas/thoughts around this?

chrisfromredfin	There's two schools around this, too - I think one is the data analysis, Top 100 stuff that should continue on the SBS Side. I think there's also like "let's get some coders together in a room and start re-working this code to get it closer to our MVP." (edited)
AmyJune (volkswagenchick)	I’ll be there. I am leading mentored contributions, we can possibly add the novice tag to some stuff to get some first time contributors? Or novices?
chrisfromredfin	That's a good thought. We should see what if anything could be tagged novice maybe a week or two in advance of the Con.
AmyJune (volkswagenchick)	If we are still in the interview phase, I can assist with that as well
leslieg	I will be onsite as well, we can add tickets for novice and also have a designated table for Project Browser. Let’s follow up on the @AmyJune (volkswagenchick)

6️⃣ Also still in dire need for someone to pick up the reins and add meeting note issues & credits. That's definitely something pretty novice as long as they can get the extension installed.

chrisfromredfin	And a great way for you to get credit even if you DIDN'T attend the meetings, because you'll have created the issue! :wink:
AmyJune (volkswagenchick)	I showed the Discover Drupal students how to do this last night, and I created one. I can do more
AmyJune (volkswagenchick)	I am expecting the students to move through some in the next couple of days.
leslieg	I was also leaving those to the Discover Drupal students for now, but can add some as needed maybe starting next week. Let me know if that fits well with the Discover Drupal timeline @AmyJune (volkswagenchick)
AmyJune (volkswagenchick)	Works great
chrisfromredfin	That's great, then I will leave it in AmyJune's capable hands to spearhead with DD students. :slightly_smiling_face:
AmyJune (volkswagenchick)	I have some travel time next week where I can get some done too… I
jjcs227	I think I can do this.... just a little bit of supervision the first couple of time.... and that's it.
AmyJune (volkswagenchick)	Are you going to be at office hours today @jjcs227?
jjcs227	Yes!
AmyJune (volkswagenchick)	Lets fire up a zoom and walk thru it. Happy to watch you :slightly_smiling_face:
jjcs227	Great!