Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track additional field in validation set for brief type #590

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

yjunechoe
Copy link
Collaborator

Summary

This PR adds a column brief_cls to agent$validation_set that tracks how the string values in the brief should be interpreted w.r.t. yaml read/write (namely, whether it had been autogenerated or not).

Example:

create_agent(~ small_table) %>%
  col_vals_gt(a, 5, brief = "a custom brief") %>%
  col_vals_gt(a, 5, brief = NULL) %>%
  el("validation_set") %>%
  subset(, c("brief", "brief_cls"))
#> # A tibble: 2 × 2
#>   brief                                         brief_cls
#>   <chr>                                         <chr>    
#> 1 "a custom brief"                              character
#> 2 "Expect that values in `a` should be > `5`. " autobrief

This is achieved by formalizing an <autobrief> class which is attached by resolve_brief() when forwarding to generate_autobriefs(). For minimal invasiveness to existing code, <autobrief> gets a methods such that indexing a vector of autobriefs perserves the class (much like behavior of <AsIs>). Since the class information is then immediately extracted out into brief_cls inside create_validation_step(), this approach allows the brief column to retain its atomic character type.

`[.autobrief` <- function(x, i, ...) structure(NextMethod("["), class = "autobrief")

Speaking of <AsIs>, this is optional but I'm also testing an experimental support for verbatim yaml to be stored/round-tripped in an agent's brief field. This should only requires a light extension to the infrastructure.

For example, it would be nice if one could pack some metadata about a validation step into a yaml and pass it to brief, after marking it with I(), like this:

create_agent(~ small_table) %>%
  col_vals_gt(a, 5, brief = I(yaml::as.yaml(list(author = "June", id = 123))))

The intended effect would be for brief to have its structure restored in the yaml representation when the agent is written out. And upon being read back into R, the brief column values would again be transformed into its flat string representation.

type: agent
tbl: ~small_table
tbl_name: ~small_table
label: '[2025-01-24|14:29:29]'
lang: en
locale: en
steps:
- col_vals_gt:
    columns: c(a, c)
    value: 5.0
    brief:
      author: June
      id: 123.0

Related GitHub Issues and PRs

Checklist

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant