Simplify AST structure (ES5)

[philipnilsson]

One idea after using this lib for a bit: How do you guys feel about changing the AST to get rid of the Infix/Assign/Pre/Postfix data types and inlining them directly into the expression data type? E.g. change

  InfixExpr a InfixOp (Expression a) (Expression a) 

to

Add a (Expression a) (Expression a)
Sub a (Expression a) (Expression a)
...

this would lead to simpler code (at least for my usage, and I believe also in the internal parser implementation), and if you do want to match on an e.g. infix operator we could provide view patterns...

I realize this probably would introduce major breakage in any existing clients, if anyone would be looking to upgrade es3 -> es5 (is this currently possible?)

[achudnov]

There's already breakage in the AST, compared to ES3. I'm not aiming to make the ES5 branch compatible with ES3 at all, and I've been advertising that 1.0 would break compatibility. So, please, don't worry about breakages (but still mind the usability and consistency).

In my work, the current shape of the AST is preferred, but if your approach is more logical, then I don't mind adopting that. I'll have to spend time porting other code for the es5 branch anyway.

What I think is necessary, though, is to make sure the AST is internally consistent and compatible with the standard. I think we've made good progress on the latter part, but the consistency currently is still to be desired.

Ultimately, it's a question of broad ASTs vs deep ASTs. For example, the infix and prefix expressions,as well as numeric literals are in the deep form: you need to look inside one of the arguments to determine exactly what kind of expression you are looking at. On the other hand, member expressions (field references) DotRef and BracketRef are in a broad form: e.g. they really denote the same thing, a field reference, but with different forms of addressing. I've been thinking of merging them into one constructor: MemberExpr a (Expression a) (Either (Id a) (Expression a)).

Before we decide on the representation, could you tell me more about view patters that you've mentioned?

[philipnilsson]

Sure, here's an example implementing a memberExpr view pattern like you just mentioned

{-# LANGUAGE ViewPatterns #-} 

memberExpr :: Expression a -> Maybe (Expression a)
memberExpr e@(DotRef _ _ _) = Just e
memberExpr e@(BracketRef _ _ _) = Just e
memberExpr _ = Nothing

foo (memberExpr -> Just e) = "member expression!"
foo (CallExpr _ _ _)       = "call expression!"
foo _                      = "other type of expression!"

Basically memberExpr is a "view" you can use in pattern matching with the -> syntax added with the ViewPatterns langauge extension. We could write similar ones for assignment expressions or prefix expressions for instance. They can expose whatever data you want, so you could even keep the existing operator data types (e.g. data InfixOp ...) and expose them through the pattern without having them in the AST.

[philipnilsson]

f (AssignmentExpr _ op exp1 exp2) = g op

could become

f (assignmentExpr -> Just op exp1 exp2) = g op

[achudnov]

Huh, that's neat. If there are view patterns, I'm fine with a wider/flatter AST, but it's got to be consistent, e.g. no discrepancies like I've outlined in my previous message. If you'd like to implement it, please, go ahead.

[achudnov]

@philipnilsson, are you working on this? I'd like to try and release what we have for es5 soon. Please, let me know if you are planning to implement this any time soon. As I said, I like your proposal very much, but I lack the necessary knowledge about the view patterns and time. I can help you port the existing code base if you like,

[philipnilsson]

I'm going to do a talk end of october which I currently will have to spend
any available extra time on, but after that I'd like to get back to this.
View patterns are quite easy to understand though, if you'd like to work on
it before that.

On Mon, Oct 14, 2013 at 10:59 PM, Andrey Chudnov
[email protected]:

@philipnilsson https://github.com/philipnilsson, are you working on
this? I'd like to try and release what we have for es5 soon. Please, let me
know if you are planning to implement this any time soon. As I said, I like
your proposal very much, but I lack the necessary knowledge about the view
patterns and time. I can help you port the existing code base if you like,

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-26288279
.

[achudnov]

Thanks for letting me know. I don't think I'll have time for studying and implementing view patterns in the next month. Moreover, there are other language-ecmascript issues of more importance that I need to attend to before I could justify spending time on this. I'll try to release 1.0 with the current syntax, but we can release the proposed new syntax as 2.0 when it's implemented and tested.

[moll]

Hey everyone, how's the ES5 parser getting along?

[achudnov]

Hi Andri. Please, see the ECMAScript 5 support milestone for the status https://github.com/jswebtools/language-ecmascript/milestone/2. Long story short: I'm trying to find some time to finish the outstanding issues. Contributions are welcome. Right now the es5 branch is pretty solid, and used in at least one project.

[moll]

Thanks for the answer! I went with parsing the ESTree-compatible JSON file that Flow outputs in Haskell for now, but will at one point look back into parsing inside Haskell, i.e. this package. ;)