Refactor ExercismGenerator to run entirely in-image, sans configlet

[bencoman]

@macta Just asking here while its on my mind without Discord access to open the discussion with you...

Before I have to dig to familiarize myself with the generation code, where in there are the generated test classes selected?

I'd like to evolve generation a bit to include a few hints in the solution class methods,
e.g. methods containing a pragma

Case in point, my proposed FlattenArray >> flattenArray: for #368

[bencoman]

Other use case... When getting back into Exercism after my big break I started as a student working through from HelloWorld. Strangely I found it confronting to be immediately hit with two "error" dialogs - one for creating the missing class and other for creating the missing method. My reaction was curious given that I knew exactly what was going on, that these were helpers were standard workflow - but with my "newcomer hat" on (per De Bono thinking hats) it "felt" like an error.

I feel strongly we need to simplify the first couple of exercises.

• For HelloWorld that means no dialogs. I remember a comment somewhere from Exercism admins that HellowWorld was meant to ensure fetch/submit work - so we could have the tests green out of the box.
• For second exercise that means class exists with missing methods, to introduce that helper dialog on its own.
• A few exercises later could missing classes to separately introduce a new helper dialog.
  i.e. gradually introducing students to our unique workflow rather than overloading them at the start.

[macta]

Good idea to look with fresh eyes. The intro documentation covers all of this with pictures, and we haven’t had anyone obviously struggle or complain yet - but it’s not a large sample as we’ve had about 50 students so far. So possibly we could revisit (although I’d hate to create more work in this area when there are lots of other parts where we need more investment - code gen is a big one you’ve identified, and equally auto mentoring and potentially the critic rules you started writing too). The decision to not include classes was partly to do with simplification as it eased supporting a dev and user environment and how we export code and configure everything - however with the move to full packages that might have lessened a bit (I recall there was also something tricky with the root Exercism package we had). We could possibly include HelloWorld and then teach them the debugger driven development in TwoFer , but I’m not totally convinced. To do this I guess we could have a known template class, and if detected, we rename it on export ... you are a mentor now right? Maybe we should ask a few newcomers if it was an issue when they submit?

…

Sent from my iPhone

On 28 May 2019, at 03:39, Ben Coman ***@***.***> wrote: Other use case... When getting back into Exercism after my big break I started as a student working through from HelloWorld. Strangely I found it confronting to be immediately hit with two "error" dialogs - one for creating the missing class and other for creating the missing method. My reaction was curious given that I knew exactly what was going on, that these were helpers were standard workflow - but with my "newcomer hat" on (per De Bono thinking hats) it "felt" like an error. I feel strongly we need to simplify the first couple of exercises. For HelloWorld that means no dialogs. I remember a comment somewhere from Exercism admins that HellowWorld was meant to ensure fetch/submit work - so we could have the tests green out of the box. For second exercise that means class exists with missing methods, to introduce that helper dialog on its own. A few exercises later could missing classes to separately introduce a new helper dialog. i.e. gradually introducing students to our unique workflow rather than overloading them at the start. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[bencoman]

Sorry, my use case redesigning early exercises distracted you from my main question (in first comment with question mark). I'll park that to follow that up in another issue.

Here I want to focus on the generation infrastructure.

Q1. Which code selects the classes to be generated?

Q2. Regarding the use of configlet...
a. whether it works on Windows to facilitate my sharing that load from you
b. being an external requirement to be setup (which to be honest has discouraged me from being more involved in generation) and a barrier to entry for future maintainers
c. after poking around your code and digesting [1] it doesn't seem hard to replicate the templating of configlet generate - ultimately the /exercises files on github just need to look the same.
d. it was good to use the official tool to begin with to eliminate unknowns, but now with greater familiarity we can consider other options

For my effort, I'd prefer to replace the configlet shell-out with pure in-Image generation rather than deal with (a.) & (b.).

[1] https://github.com/exercism/docs/blob/master/language-tracks/exercises/anatomy/readmes.md#generating-a-readme

[macta]

A - there are two generators , to clarify - ExercismGenerator creates the stubs needed for readme.md files as well as exports code into the required places that Exercism expects. (It then gets included when you commit your changes - although iceberg doesn’t like commuting code outside it’s src dir, so you have to use the cmd line in a second commit). Unrelated - ExercismExerciseGenerator, reads the problem definition files to create WIP Exercises, that we have to solve (and can then submit to students - the rule is, we need to pass the tests ourselves first) I think it’s the first one you are referring to - and as we have become less and less dependent on the cli tools of Exercism, we can certainly replace it. HOWEVER - word of caution (and this has held me back so far), if they change the format or structure of Exercism, we won’t be insulated from that change - which is why is I opted to play along with their tools while we learned how it worked. However it’s quite trivial - #generateTo: asks for all the exercises and iterates over them, it pulls out the hint from the comment and then calls configlet at the end. Ages ago, we didn’t store the problem description in the test comment, and when we did that, I felt it was easier to pop the hint at the end of that comment, so really we only need the template instructions to go at the end , and they are in a text file (in /config) that we can just append to that text - so it’s a trivial change. I doubt we need to use a template engine either - maybe just the normal String #macros method would be fine? Related to this, ExercismConfigGenerator then iterates all the exercises and rewrites the config.json that Exercism uses to display exercises on the website. It has some tests, pretty straight forward, do them in pre-req order, with non core alphabetically. It doesn’t do any sense checking, so Travis may later complain if a core exercise pre-reqs a non core one etc (this is a configlet check in the build script - but it’s fine running in the CI). The other generator is a tiny bit more involved - I can explain that another time (but essentially it expects you to check out the problem-definitions repo, and then you point to that directory and it iterates over the exercism directories and parses the Json to infer the test names). Not rocket science, but the definitions are quite weak, which is where it’s a bit trickier (there is a recent push to add some meta tags to help inferencing). It’s a spike that got us going, but it needs rework (and tests) to handle regeneration and customise the generation (given the weak specs) Tim

…

Sent from my iPhone

On 29 May 2019, at 03:49, Ben Coman ***@***.***> wrote: Sorry, my use case redesigning early exercises distracted you from my main question (in first comment with question mark). I'll park that to follow that up in another issue. Here I want to focus on the generation infrastructure. Q1. Which code selects the classes to be generated? Q2. Regarding the use of configlet... a. whether it works on Windows to facilitate my sharing that load from you b. being an external requirement to be setup (which to be honest has discouraged me from being more involved in generation) and a barrier to entry for future maintainers c. after poking around your code and digesting [1] it doesn't seem hard to replicate the templating of configlet generate - ultimately the /exercises files on github just need to look the same. d. it was good to use the official tool to begin with to eliminate unknowns, but now with greater familiarity we can consider other options For my effort, I'd prefer to replace the configlet shell-out with pure in-Image generation rather than deal with (a.) & (b.). [1] https://github.com/exercism/docs/blob/master/language-tracks/exercises/anatomy/readmes.md#generating-a-readme — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[bencoman]

I think it’s the first one [ExercismGenerator] you are referring to

Yes. Glad to hear you concur with my guess its trivial to replace. I'll have a go at it.

if they change the format or structure of Exercism, we won’t be insulated from that change

I'm foresaw that also, but think its a reasonable trade off. They'd have to announcement it, at least for users to manually update the configlet, and then it can run be from the command line to compare and implement the difference.

[macta]

yes - having brain dumped, its a good thing to do - particularly to simplify and remove some dependencies.

[bencoman]

What is the importance of this...?

    self halt: 'Forgot to update Exercise hint for: ', anExercismExercise name ].

[macta]

It probably should be self error - I think I was thinking you might correct the comment and press resume (but in practice I’ve been rerunning the whole operation). The idea is that every exercise should have a useful hint and not the placeholder TBD. I considered a separate method for that text, but it’s useful to consider what to write in the context of the problem text. It does make regeneration of exercises a bit harder though - but as there is a method to extract the hint , probably fine?

…

Sent from my iPhone

On 31 May 2019, at 05:02, Ben Coman ***@***.***> wrote: What is the importance of this...? self halt: 'Forgot to update Exercise hint for: ', anExercismExercise name ]. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[bencoman]

@macta, Curious why #generateTestMethodsOn:calling:using:prefix: is calling recursively. My first thought is the multiple level of cases keys might be inconsistent across problem specifications.
[Edit] Ahh, I see the canonical-schema.json is recursive on testGroup. Well Thats Fantastic.

[bencoman]

In #generateTestNamed:in:variable:selector:parameters:expecting: why are line ending #cflf?

That seems Windows platform specific. I wasn't sensitive to this last week but following on from Sven's comment in the mail list, #cr is used internal to Pharo and conversion done per platform when its written to disk.