Move to an AST like data structure #76

Merged

Aire-One merged 39 commits from feat/refactor-ast into master 2023-08-03 21:03:16 +02:00

Conversation 4 Commits 39 Files Changed 29 +1920 -367

Aire-One commented 2023-01-29 19:07:01 +01:00

Owner

Copy Link

The objective of this work is to move away from the current "Entity" stuff. I think we are reaching the limits of this system for some time now, and the recent issues from #74 show that we need to write visitors to apply manual fixes on the scraping result. Visitor will be really easy to write with a data structure based on a tree.

We use some new features from recent Teal releases, so this branch is based on top of feat/fixes-teal-0.15.0 (#75).

With these changes, we are also slowly moving away from Penlight data structure (mainly pl.List and pl.Map) and some part of the code base are being rewritten with what I call "a better Teal feeling". Let's see where it brings us!

Fixes: #57, #59, #74

The objective of this work is to move away from the current "Entity" stuff. I think we are reaching the limits of this system for some time now, and the recent issues from #74 show that we need to write visitors to apply manual fixes on the scraping result. Visitor will be really easy to write with a data structure based on a tree. We use some new features from recent Teal releases, so this branch is based on top of `feat/fixes-teal-0.15.0` (#75). With these changes, we are also slowly moving away from Penlight data structure (mainly `pl.List` and `pl.Map`) and some part of the code base are being rewritten with what I call "a better _Teal feeling_". Let's see where it brings us! Fixes: #57, #59, #74

Aire-One added 4 commits 2023-01-29 19:07:04 +01:00

8d389bca21 fix(utils): `do_or_fail` `res` need to be promoted

df379a4537 fix(entity): can't infer record with record

37f0d6bac4 fix(generator): can't infer generic type

e6cd4712fa fix(entity): shouldn't redefine signature

Aire-One added 1 commit 2023-01-29 19:19:29 +01:00

ee7f986465 feat(types): introduce new data structures

Aire-One added 1 commit 2023-01-29 19:21:54 +01:00

2017fffb48 feat(scraper): move "Object properties" to AST

Aire-One added 1 commit 2023-01-29 19:22:29 +01:00

34237dbcae run: some play with the entrypoint

Aire-One added 1 commit 2023-01-29 19:35:04 +01:00

1f5974e2a8 fixup! feat(scraper): move "Object properties" to AST

Aire-One commented 2023-02-04 00:25:35 +01:00

Author

Owner

Copy Link

With #75 merged, I can now rebase to have a correct CI output ✨

With #75 merged, I can now rebase to have a correct CI output ✨

Aire-One force-pushed feat/refactor-ast from 1f5974e2a8 to 44de7a23df 2023-02-04 00:26:31 +01:00 Compare

Aire-One added 1 commit 2023-02-04 00:40:50 +01:00

e9dbc27770 feat(scraper): move "Signals" to AST

Aire-One added 1 commit 2023-02-04 00:41:29 +01:00

daed32976a chore: add cSpell wording

Aire-One added 1 commit 2023-02-11 18:33:06 +01:00

11721a3331 fix(types): Dag requires path

Aire-One added 1 commit 2023-04-12 00:20:03 +02:00

de8c7c682b feat(scraper): implement all `section_scrapers`

Aire-One commented 2023-04-12 00:38:12 +02:00

Author

Owner

Copy Link

Finally, coming back at it! I have finished to port the scraper to generate the new tree structure. I'm pretty happy with how it ends up 😄

Finally, coming back at it! I have finished to port the scraper to generate the new tree structure. I'm pretty happy with how it ends up 😄

Aire-One added 1 commit 2023-04-16 12:54:46 +02:00

d4374fd989 feat(scraper): function can populate `other_nodes` ...

To allow `extract_section_functions`  to populate `other_node`, we need
to use the `module_name` and migrate the function name scrap method to
use the actually displayed text instead of playing with the node `name`
attribute.

Because of this change, we also need to update
`extract_section_variables`. Not a big deal. BTW, we resolved an issue
where `string` variable can wrongly be detected as `enum`.

Aire-One added 1 commit 2023-04-16 22:15:58 +02:00

48f9d3f3ca feat(generator): base implem for `generate_teal`

Aire-One commented 2023-04-16 22:27:39 +02:00

Author

Owner

Copy Link

The base implementation for the generator came together pretty quickly. It needs more work, but I'm super happy with the direction it takes.

Something that worries me now is that I have to adjust some stuffs on both sides now, the scraper and the generator, to get correct Teal types definitions in the output. Every change can introduce regressions. I need to get back to #24 and find a proper way to use Busted with Teal. So this PR will (probably) have to survive another freeze 😢 (or I'm not motived enough to finish the Busted thing and decide to continue to work here... Who knows?)

The base implementation for the generator came together pretty quickly. It needs more work, but I'm super happy with the direction it takes. Something that worries me now is that I have to adjust some stuffs on both sides now, the scraper and the generator, to get correct Teal types definitions in the output. Every change can introduce regressions. I need to get back to #24 and find a proper way to use Busted with Teal. So this PR will (probably) have to survive another freeze 😢 (or I'm not motived enough to finish the Busted thing and decide to continue to work here... Who knows?)

Aire-One referenced this pull request 2023-04-20 18:16:57 +02:00

Properly handle CAPI vs Lua libraries #59

Aire-One force-pushed feat/refactor-ast from 48f9d3f3ca to 60bbc3d6d4 2023-04-24 03:00:16 +02:00 Compare

Aire-One force-pushed feat/refactor-ast from 60bbc3d6d4 to 592e62d6fa 2023-05-01 16:46:24 +02:00 Compare

Aire-One referenced this pull request 2023-05-04 00:26:06 +02:00

Some `enum`s aren't parsed correctly #57

Aire-One added 1 commit 2023-05-04 00:45:48 +02:00

5d45637954 spec(module_doc): add basic test

Aire-One added 1 commit 2023-05-04 00:46:41 +02:00

71f43ecb9b spec(module_doc): add "Signals" test

Aire-One force-pushed feat/refactor-ast from 71f43ecb9b to 1faa35971e 2023-05-04 00:52:55 +02:00 Compare

Aire-One force-pushed feat/refactor-ast from 1faa35971e to 129f15ea9d 2023-05-04 01:02:03 +02:00 Compare

Aire-One added 1 commit 2023-05-05 00:04:35 +02:00

9165e98db3 chore: configure lua-local debugger for Teal 🧙

Aire-One added 1 commit 2023-05-05 00:41:10 +02:00

5e6ceb7e04 spec(module_doc): add "Static module functions" tests

Aire-One added 1 commit 2023-05-08 14:25:59 +02:00

efb54bf92b spec(module_doc): add "Object methods" tests

Aire-One added 1 commit 2023-05-08 23:12:25 +02:00

c09d9d15f0 build: add coverage command

Aire-One force-pushed feat/refactor-ast from c09d9d15f0 to 3bf9ac4ebd 2023-05-08 23:14:27 +02:00 Compare

Aire-One added 1 commit 2023-05-08 23:25:10 +02:00

56aa489558 chore: rename Generator -> Printer

Aire-One added 1 commit 2023-05-08 23:27:45 +02:00

09716bb597 fixup! chore: rename Generator -> Printer

Aire-One added 1 commit 2023-05-08 23:37:50 +02:00

6d647cd9bc fixup! chore: rename Generator -> Printer

Aire-One added 1 commit 2023-05-08 23:48:18 +02:00

f324e9e303 chore(spec): defined expected with type safety

Aire-One added 1 commit 2023-05-08 23:50:52 +02:00

9fdbbe2096 fixup! chore(spec): defined expected with type safety

Aire-One added 1 commit 2023-05-09 00:27:37 +02:00

106a456dd0 spec(teal_type_definition): add basic test

Aire-One added 1 commit 2023-05-09 00:29:34 +02:00

4eeb8894d2 chore(just): add pattern parameter to test command

Aire-One added 1 commit 2023-05-09 00:41:02 +02:00

76b9d5032e spec(teal_type_definition): test Enum node

Aire-One added 1 commit 2023-05-12 23:58:16 +02:00

136560aad6 fixup! chore(just): add pattern parameter to test command

Aire-One added 1 commit 2023-05-13 00:19:58 +02:00

5ae73bbbc6 spec(teal_type_definition): test Variable node

Aire-One added 1 commit 2023-05-13 00:25:20 +02:00

7b08f10150 spec(teal_type_definition): test Function node

Aire-One added 1 commit 2023-05-13 00:29:59 +02:00

eee6a47fbe spec(teal_type_definition): test Record node

Aire-One added 1 commit 2023-05-13 03:42:37 +02:00

51e5d4617f fix(module_doc): named parameter table detection

Aire-One added 1 commit 2023-05-13 14:33:19 +02:00

c0ed77aff5 feat(printer): indent generated code

Aire-One added 1 commit 2023-05-14 13:00:38 +02:00

0920e49335 fixup! fixup! chore(just): add pattern parameter to test command

Aire-One added 1 commit 2023-05-14 13:05:14 +02:00

a26dce4e08 chore: update the `Debug spec` command

Aire-One added 1 commit 2023-05-20 15:14:10 +02:00

0016e55cb7 feat(Visitors): implement type_coercion

Aire-One added 1 commit 2023-05-21 20:00:18 +02:00

5de8e105a7 feat(Visitors): implement module_dependencies

Aire-One added 1 commit 2023-05-21 20:17:45 +02:00

78d50c7541 feat(printer): render `require` statements

Aire-One added 1 commit 2023-05-21 20:42:41 +02:00

c5430c59b7 spec(module_doc): fix and refactor ...

Fix the tests by adding the new keys from `"module"` tokens.

We now use a `test` function to make the `it` easier to write/read.

Aire-One added 1 commit 2023-07-18 20:25:10 +02:00

d4d0f387a0 fix(spec): ASTs need to have `dependencies`

Aire-One force-pushed feat/refactor-ast from d4d0f387a0 to eec274af44 2023-07-18 20:59:24 +02:00 Compare

Aire-One added 1 commit 2023-07-28 01:10:50 +02:00

160a092896 spec(teal_type_definition): test module dependencies

Aire-One added 1 commit 2023-07-31 19:02:17 +02:00

588296abcd fix(visitor): module_dependencies renames dependency usages

Aire-One added 1 commit 2023-07-31 19:02:53 +02:00

be59debe82 fix(visitor): module_dependencies shouldn't self require

Aire-One added 1 commit 2023-07-31 19:20:43 +02:00

a0d4765b91 fix(scraper): rename constructor methods to `new`

Aire-One added 1 commit 2023-07-31 19:26:54 +02:00

b911cab58d feat(printer): teal_type_definition should use the `require type` notation

Aire-One added 1 commit 2023-07-31 19:49:23 +02:00

86ebdac0d9 fixup! feat(Visitors): implement type_mapping

Aire-One added 1 commit 2023-08-01 23:22:53 +02:00

3680c1ebb5 run: let's make the main entry point do something

Aire-One added 1 commit 2023-08-01 23:36:55 +02:00

c8ff0aaaec fix(visitor/type_mapping): update mapping

Aire-One force-pushed feat/refactor-ast from c8ff0aaaec to 73f6074ef2 2023-08-01 23:41:50 +02:00 Compare

Aire-One commented 2023-08-01 23:43:24 +02:00

Author

Owner

Copy Link

With this final rebase, we should finally be ready to merge! 🎉

With this final rebase, we should finally be ready to merge! 🎉

Aire-One changed title from WIP: Move to an AST like data structure to Move to an AST like data structure 2023-08-01 23:43:37 +02:00

Aire-One merged commit 4dd8d433f4 into master 2023-08-03 21:03:16 +02:00

Aire-One referenced this issue from a commit 2023-08-03 21:03:17 +02:00

Merge pull request 'Move to an AST like data structure' (#76) from feat/refactor-ast into master

Aire-One deleted branch feat/refactor-ast 2023-08-03 21:03:19 +02:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

blocked

  

ci

  

deps

  

duplicate

No Label

blocked

ci

deps

duplicate

Milestone

Clear milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

No Assignees

1 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Aire-One/awesomewm.d.tl#76

No description provided.