Inconsistent white space and new line on windows and linux #27

sunshine69 · 2024-11-21T21:09:07Z

Hi there,

I just realize that the same config and template (that is trimblock is true and lstrip block is true, on linux it produces the expected behaviour but on windows it seems to ignore. Not sure if I made anything crazy ...

The template file content is like this

{% for line in lines %}
{{ line }}
{% endfor %}
New line

On windows it produces


line1

line2

line3

New line

On linux is expected

line1
line2
line3
New line

The text was updated successfully, but these errors were encountered:

NikolaLohinski · 2024-11-23T16:06:18Z

Hey @sunshine69 ! Thanks for reporting ;

I pushed 8d56e4e as an attempted fix but I don't have a windows machine already setup and ready for testing this provider. I'll get that and confirm it works as expected or figure out the right fix in the next days

Cheers

sunshine69 · 2024-11-24T05:37:05Z

Great, in my automatin-go I got this file with content

#jinja2:variable_start_string:'{$', variable_end_string:'$}', trim_blocks:True, lstrip_blocks:True
This has config line
{$ header $}
{% for l in lines -%}
{$ l $}
{% endfor -%}

The template output are identical for windows and linux now. I am not sure why there is another new line at the end in both cases but the initial problem has been fixed.

This has config line
Header
line1
line2
line3

Thank you very much for prompt fixes. Highly appreciated it.

Kind regards

sunshine69 · 2024-11-24T05:47:44Z

Interesting behaviour

The above code run same on windows and linux which does not happen before but now fixed on windows. However these

#jinja2:variable_start_string:'{$', variable_end_string:'$}', trim_blocks:True, lstrip_blocks:True
This has config line
{$ header $}
{% for l in lines %}
{$ l $}
{% endfor %}

produce inconsistent still

On linux it is the same (which is expected as well if option like the first line set correctly

But on windows I see

This has config line
Header

line1

line2

line3

So trimblock option seems to not working on windows correctly? At least I can use the dash to adjust but not sure.

NikolaLohinski · 2024-11-24T15:42:07Z

Hmm seems I have missed some white space handling logic somewhere in the code with my last fix ... no worries, I'll figure it out once I have setup my windows machine and can run tests on it. Bear with me and thanks for the report again 🙇

sunshine69 · 2024-11-24T23:00:45Z

No worries. Normally I adjust white spaces using dash - anyway which is like more explicit and better. It works for me now. I will leave it open if you still want to investigate.

#27

NikolaLohinski · 2024-11-25T08:19:51Z

Hey @sunshine69 I added b03bd7c and tested that on windows, and it seems to be working just fine for me . You sure you're version of gonja is up to date ?

Would you mind testing again on your side 🙏 ?

Clone the repo
Install ginkgo : https://onsi.github.io/ginkgo/#installing-ginkgo
run
```
ginkgo run -focus issues/27 ./...
```

sunshine69 · 2024-11-25T22:22:33Z

I will start testing now, will update here soon. The previous one already works for both if control white space manually using - , just the option trim_block on windows seems not to yet. Will login to work computer and look

sunshine69 · 2024-11-26T10:00:42Z

OK I have run as instructed (not understand ginkgo whatever yet lol) and it seems to be passed

Ginkgo ran 7 suites in 1m1.9305103s
Test Suite Passed

However mine test is still seeing it is not the same as on linux. Might be my code. I will have a look myself.

In my library I added the ability to read the config at the first line and it might be broken. If you trusted ginkgo test pass is good please close the ticket.

Thanks a lot

sunshine69 · 2024-11-26T11:06:50Z

I must be crazy. I could not prove that you BDD test is faulty but I could not make my code works. No matter what; added print debug, dump the config; everything looks fine. Just when rendering on windows the trim block options does nothing.

It must be something somewhere mysteriously. I am a bit tired after long work day; so will leave it now as is.

NikolaLohinski · 2024-11-26T20:13:30Z

@sunshine69 Might be related to the go version you are using 🤷

sunshine69 · 2024-11-28T22:00:35Z

I got a crash in line /home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/exec/renderer.go:64 +0x6e5

if n.RemoveFirstLineReturn && (output[0:1] == "\n" || output[0:2] == "\r\n") {

Looks we have to check the slice length?

--- FAIL: TestJinja2 (0.05s)
panic: runtime error: slice bounds out of range [:2] with length 1 [recovered]
	panic: runtime error: slice bounds out of range [:2] with length 1

goroutine 7 [running]:
testing.tRunner.func1.2({0x881ea0, 0xc00002a408})
	/mnt/live/memory/images/002-ubuntu-devtool-x86_64.xzm/opt/go/src/testing/testing.go:1632 +0x230
testing.tRunner.func1()
	/mnt/live/memory/images/002-ubuntu-devtool-x86_64.xzm/opt/go/src/testing/testing.go:1635 +0x35e
panic({0x881ea0?, 0xc00002a408?})
	/mnt/live/memory/images/002-ubuntu-devtool-x86_64.xzm/opt/go/src/runtime/panic.go:785 +0x132
github.com/nikolalohinski/gonja/v2/exec.(*Renderer).Visit(0xc0001af0c0, {0x964920?, 0xc0000398f0})
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/exec/renderer.go:64 +0x6e5
github.com/nikolalohinski/gonja/v2/nodes.Walk({0x962f40?, 0xc0001af0c0?}, {0x964920, 0xc0000398f0})
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/nodes/walk.go:12 +0x38
github.com/nikolalohinski/gonja/v2/nodes.Walk({0x962f40?, 0xc0001af0c0?}, {0x9648d0, 0xc0001aea80})
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/nodes/walk.go:29 +0x167
github.com/nikolalohinski/gonja/v2/exec.(*Renderer).ExecuteWrapper(0xc0000452e0?, 0xc0001aea80)
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/exec/renderer.go:127 +0x33
github.com/nikolalohinski/gonja/v2/builtins/control_structures.(*IfControlStructure).Execute(0xc0001aea40, 0xc0001af080, 0xc0001a40d9?)
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/builtins/control_structures/if.go:36 +0xe5
github.com/nikolalohinski/gonja/v2/exec.(*Renderer).Visit(0xc0001af080, {0x964970?, 0xc0001b3470})
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/exec/renderer.go:115 +0x702
github.com/nikolalohinski/gonja/v2/nodes.Walk({0x962f40?, 0xc0001af080?}, {0x964970, 0xc0001b3470})
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/nodes/walk.go:12 +0x38
github.com/nikolalohinski/gonja/v2/nodes.Walk({0x962f40?, 0xc0001af080?}, {0x9648d0, 0xc0001ae240})
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/nodes/walk.go:29 +0x167
github.com/nikolalohinski/gonja/v2/exec.(*Renderer).ExecuteWrapper(0xc0000128e8?, 0xc0001ae240)
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/exec/renderer.go:127 +0x33
github.com/nikolalohinski/gonja/v2/builtins/control_structures.(*ForControlStructure).Execute(0xc0000c09b0, 0xc0001aef40, 0xc0000150d0?)
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/builtins/control_structures/for.go:158 +0x652
github.com/nikolalohinski/gonja/v2/exec.(*Renderer).Visit(0xc0001aef40, {0x964970?, 0xc0001b34a0})
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/exec/renderer.go:115 +0x702
github.com/nikolalohinski/gonja/v2/nodes.Walk({0x962f40?, 0xc0001aef40?}, {0x964970, 0xc0001b34a0})
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/nodes/walk.go:12 +0x38
github.com/nikolalohinski/gonja/v2/nodes.Walk({0x962f40?, 0xc0001aef40?}, {0x9648a8, 0xc00007fe40})
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/nodes/walk.go:23 +0x1b2
github.com/nikolalohinski/gonja/v2/exec.(*Renderer).Execute(...)
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/exec/renderer.go:137
github.com/nikolalohinski/gonja/v2/exec.(*Template).Execute(0xc0000c08c0, {0x9624a0, 0xc0001b34d0}, 0xc00007fd00?)
	/home/stevek/go/pkg/mod/github.com/nikolalohinski/gonja/[email protected]/exec/template.go:73 +0x225
github.com/nikolalohinski/gonja/v2/exec.(*Template).ExecuteToString(0xc0000c08c0, 0xc0001bf9d0)

Now not sure why on this machine it happened but not on other at least not yet seeing it. Let me commit it and pull on the work machine for the same test case and see.

golang version is latest

go version
go version go1.23.3 linux/amd64

sunshine69 · 2024-11-29T12:38:47Z

OK I add a line into the test and it crash. Please have a look at the diff
https://github.com/sunshine69/gonja/blob/e460dfe8e9ece10f80d47e84d8fda250b4cb757c/tests/integration/config_test.go#L312

The crash log is bellow

ginkgo run -focus issues/27 ./...
[1732883718] pystring-methods - 0/381 specs SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS SUCCESS! 32.625021ms PASS
time="2024-11-29T22:35:41+10:00" level=error msg="Value.String() not implemented for type: func\n"
[1732883718] exec - 0/113 specs SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS SUCCESS! 31.134354ms PASS
[1732883718] loader - 0/21 specs SSSSSSSSSSSSSSSSSSSSS SUCCESS! 1.886889ms PASS
[1732883718] parser - 0/70 specs SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS SUCCESS! 8.619648ms PASS
[1732883718] integration - 1/133 specs SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
• [PANICKED] [0.002 seconds]
config [JustBeforeEach] https://github.com/NikolaLohinski/gonja/issues/27 should return the expected rendered content
  [JustBeforeEach] /home/stevek/src/gonja/tests/integration/config_test.go:33
  [It] /home/stevek/src/gonja/tests/integration/config_test.go:321

  [PANICKED] Test Panicked
  In [JustBeforeEach] at: /mnt/live/memory/images/002-ubuntu-devtool-x86_64.xzm/opt/go/src/runtime/panic.go:129 @ 11/29/24 22:35:42.033

  runtime error: slice bounds out of range [:2] with length 1

  Full Stack Trace
    github.com/nikolalohinski/gonja/v2/exec.(*Renderer).Visit(0xc000322d00, {0xbf2f40?, 0xc0003121a0})
        /home/stevek/src/gonja/exec/renderer.go:64 +0x6e5
    github.com/nikolalohinski/gonja/v2/nodes.Walk({0xbf07c0?, 0xc000322d00?}, {0xbf2f40, 0xc0003121a0})
        /home/stevek/src/gonja/nodes/walk.go:12 +0x38
    github.com/nikolalohinski/gonja/v2/nodes.Walk({0xbf07c0?, 0xc000322d00?}, {0xbf2ef0, 0xc000322680})
        /home/stevek/src/gonja/nodes/walk.go:29 +0x167
    github.com/nikolalohinski/gonja/v2/exec.(*Renderer).ExecuteWrapper(0xc00031c340?, 0xc000322680)
        /home/stevek/src/gonja/exec/renderer.go:127 +0x33
    github.com/nikolalohinski/gonja/v2/builtins/control_structures.(*IfControlStructure).Execute(0xc000322640, 0xc000322cc0, 0xaebe27?)
        /home/stevek/src/gonja/builtins/control_structures/if.go:36 +0xe5
    github.com/nikolalohinski/gonja/v2/exec.(*Renderer).Visit(0xc000322cc0, {0xbf2f90?, 0xc0004cacc0})
        /home/stevek/src/gonja/exec/renderer.go:115 +0x702
    github.com/nikolalohinski/gonja/v2/nodes.Walk({0xbf07c0?, 0xc000322cc0?}, {0xbf2f90, 0xc0004cacc0})
        /home/stevek/src/gonja/nodes/walk.go:12 +0x38
    github.com/nikolalohinski/gonja/v2/nodes.Walk({0xbf07c0?, 0xc000322cc0?}, {0xbf2ef0, 0xc00007f140})
        /home/stevek/src/gonja/nodes/walk.go:29 +0x167
    github.com/nikolalohinski/gonja/v2/exec.(*Renderer).ExecuteWrapper(0xc00019e1b0?, 0xc00007f140)
        /home/stevek/src/gonja/exec/renderer.go:127 +0x33
    github.com/nikolalohinski/gonja/v2/builtins/control_structures.(*ForControlStructure).Execute(0xc0000ea780, 0xc000322b80, 0xc000194018?)
        /home/stevek/src/gonja/builtins/control_structures/for.go:158 +0x652
    github.com/nikolalohinski/gonja/v2/exec.(*Renderer).Visit(0xc000322b80, {0xbf2f90?, 0xc0004cacf0})
        /home/stevek/src/gonja/exec/renderer.go:115 +0x702
    github.com/nikolalohinski/gonja/v2/nodes.Walk({0xbf07c0?, 0xc000322b80?}, {0xbf2f90, 0xc0004cacf0})
        /home/stevek/src/gonja/nodes/walk.go:12 +0x38
    github.com/nikolalohinski/gonja/v2/nodes.Walk({0xbf07c0?, 0xc000322b80?}, {0xbf2ec8, 0xc00007eb40})
        /home/stevek/src/gonja/nodes/walk.go:23 +0x1b2
    github.com/nikolalohinski/gonja/v2/exec.(*Renderer).Execute(...)
        /home/stevek/src/gonja/exec/renderer.go:137
    github.com/nikolalohinski/gonja/v2/exec.(*Template).Execute(0xc0000ea690, {0xbf0340, 0xc0004cad20}, 0xc0000911a0?)
        /home/stevek/src/gonja/exec/template.go:73 +0x225
    github.com/nikolalohinski/gonja/v2/exec.(*Template).ExecuteToString(0xc0000ea690, 0x0)
        /home/stevek/src/gonja/exec/template.go:85 +0x97
    github.com/nikolalohinski/gonja/v2/tests/integration_test.init.func2.2()
        /home/stevek/src/gonja/tests/integration/config_test.go:39 +0x92
------------------------------
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS

Summarizing 1 Failure:
  [PANICKED!] config [JustBeforeEach] https://github.com/NikolaLohinski/gonja/issues/27 should return the expected rendered content
  /mnt/live/memory/images/002-ubuntu-devtool-x86_64.xzm/opt/go/src/runtime/panic.go:129

Ran 1 of 133 Specs in 0.016 seconds
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 132 Skipped
--- FAIL: TestLegacy (0.03s)
FAIL

Ginkgo ran 7 suites in 23.486311348s

There were failures detected in the following suites:
  integration ./tests/integration

Test Suite Failed

sunshine69 closed this as completed Nov 24, 2024

sunshine69 reopened this Nov 24, 2024

NikolaLohinski added a commit that referenced this issue Nov 25, 2024

chore(tests): add a test to cover issue 27

b03bd7c

#27

sunshine69 closed this as completed Nov 26, 2024

sunshine69 reopened this Nov 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Inconsistent white space and new line on windows and linux #27

Inconsistent white space and new line on windows and linux #27

sunshine69 commented Nov 21, 2024

NikolaLohinski commented Nov 23, 2024 •

edited

Loading

sunshine69 commented Nov 24, 2024

sunshine69 commented Nov 24, 2024

NikolaLohinski commented Nov 24, 2024

sunshine69 commented Nov 24, 2024 •

edited

Loading

NikolaLohinski commented Nov 25, 2024 •

edited

Loading

sunshine69 commented Nov 25, 2024

sunshine69 commented Nov 26, 2024

sunshine69 commented Nov 26, 2024 •

edited

Loading

NikolaLohinski commented Nov 26, 2024

sunshine69 commented Nov 28, 2024

sunshine69 commented Nov 29, 2024

Inconsistent white space and new line on windows and linux #27

Inconsistent white space and new line on windows and linux #27

Comments

sunshine69 commented Nov 21, 2024

NikolaLohinski commented Nov 23, 2024 • edited Loading

sunshine69 commented Nov 24, 2024

sunshine69 commented Nov 24, 2024

NikolaLohinski commented Nov 24, 2024

sunshine69 commented Nov 24, 2024 • edited Loading

NikolaLohinski commented Nov 25, 2024 • edited Loading

sunshine69 commented Nov 25, 2024

sunshine69 commented Nov 26, 2024

sunshine69 commented Nov 26, 2024 • edited Loading

NikolaLohinski commented Nov 26, 2024

sunshine69 commented Nov 28, 2024

sunshine69 commented Nov 29, 2024

NikolaLohinski commented Nov 23, 2024 •

edited

Loading

sunshine69 commented Nov 24, 2024 •

edited

Loading

NikolaLohinski commented Nov 25, 2024 •

edited

Loading

sunshine69 commented Nov 26, 2024 •

edited

Loading