Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CN] Syntax for configuring testing on a per-function and/or per-predicate basis #816

Open
ZippeyKeys12 opened this issue Jan 8, 2025 · 2 comments
Labels
CN spec testing cn enhancement New feature or request language Related to design of the CN language

Comments

@ZippeyKeys12
Copy link
Collaborator

CN-Test-Gen currently has many CLI options for tuning the input generators and tests. However, many of these options make more sense on a per-function or per-predicate basis, than on a per-file basis.

For some simple examples of test options, one might want a particularly important function to be tested with more or larger inputs. Instead of wanting a certain number of inputs, one might also want to test a function for a certain amount of time.

The way our input generators are derived, they correspond to either a predicate or specification. So all options for tuning the generators would be useful to be able to control per-specification and per-predicate.

For functions, one could add tested {...}, similar to trusted, where a record is given containing the options. Not sure what the per-predicate version would look like.

@ZippeyKeys12 ZippeyKeys12 added enhancement New feature or request cn language Related to design of the CN language CN spec testing labels Jan 8, 2025
@bcpierce00
Copy link
Collaborator

Relatedly, we might want to consider introducing syntax for recording individual (generated or hand-written) tests as unit tests.

@yav
Copy link
Collaborator

yav commented Jan 8, 2025

While we think about this, we may want to consider ways to specify some of these things in a separate file (in addition to some inline notation).

The benefit of specifying things in an external file is that the code is not cluttered with meta data, and you get the ability to have multiple settings for the same code (e.g., "quick test" to do during development vs "long test" that you do during CI)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CN spec testing cn enhancement New feature or request language Related to design of the CN language
Projects
None yet
Development

No branches or pull requests

3 participants