Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qmstr.yaml configuration instructions #469

Open
utkarshyen opened this issue Mar 11, 2020 · 11 comments
Open

qmstr.yaml configuration instructions #469

utkarshyen opened this issue Mar 11, 2020 · 11 comments
Milestone

Comments

@utkarshyen
Copy link

utkarshyen commented Mar 11, 2020

Hello @mirkoboehm

I have followed the installation steps mentioned in document.
Tested the tutorials, debian curl and json-c packages.

However, I could not see the binary licenses (target licenses) in scancode.json file, even in report I could see all the tabs as BLANK.

Please find the qmstr.yaml file.

NOTE I have used same qmstr.yaml file from json-c tutorials for testing purpose.


project:
name: "elfutils"
metadata:
Vendor: ""
OcFossLiaison: ""
OcComplianceContact: ""
analysis:
- analyzer: spdx-identifier-analyzer
name: "Simple SPDX Analyzer"
config:
workdir: "/buildroot"
- analyzer: scancode-analyzer
name: "Scancode Analyzer"
config:
workdir: "/buildroot"
resultfile: "/buildroot/scancode.json"
#cached: "true"
- analyzer: test-analyzer
name: "Simple CI Test Analyzer"
config:
workdir: "/buildroot"
reporting:
- reporter: test-reporter
name: "Test Reporter"
config:
siteprovider: "Endocode"
- reporter: package-manifest-reporter
name: "Package manifest Reporter"
config:
outputdir: "/buildroot"
pathSubst: ".+?/.+?/.+?/(.+)||./$1"
- reporter: qmstr-reporter-html
name: "Public HTML Reports"
config:
siteprovider: "Endocode"
baseurl: "http://qmstr.org/packages/"

Can you please put some light on this? Am I missing any steps here? Or any configuration setting?
Also, https://qmstr.org/documentation/introduction/getting-started/Qmstr.yaml.md page is not found.

Regards,
UT

@utkarshyen
Copy link
Author

@senaingrid Can you please provide your views on this?

@utkarshyen
Copy link
Author

Any update on this?

@fullsushidev fullsushidev self-assigned this Mar 17, 2020
@fullsushidev fullsushidev added the question Further information is requested label Mar 17, 2020
@fullsushidev
Copy link
Contributor

Hi @utkarshyen, which document are you following? It seems you're using an outdated version of the config file. You can find the yaml example file at this link: https://qmstr.org/documentation/configuration/

Best,
Ingrid

@utkarshyen
Copy link
Author

Hello @senaingrid

I followed the same document which you mentioned for config file

https://qmstr.org/documentation/configuration/

I'm trying to look the issue at my end. I could not see any entries in generated HTML report.
Also, some of the analyzers are failing for some of the packages.

Regards,
UT

@fullsushidev
Copy link
Contributor

Are you using a public fork of elfutils? Can you share the link so I can try to replicate the scenario here?

@utkarshyen
Copy link
Author

I'm using ubuntu 18.04 debian source,
"elfutils-0.170", building it with make instead of debian build (dpkg-buildpackage) framework.

Regards,

@GiasemiSh
Copy link

GiasemiSh commented Mar 18, 2020

Hi @utkarshyen. Sorry in advance, our documentation is not complete but we are trying our best.

You tried the tutorial, debian curl and json-c packages. Did they work for you?
You mention that you could not see the binary licenses. Was the scancode.json empty?

@utkarshyen
Copy link
Author

Hello @GiasemiSh,

Sorry to bother.
Yes. I tried both the tutorials debian curl and json-c packages. However,
debian curl package - analyze fails for spdx-identifier-analyzer as well as for scancode-analyzer.
Thus no report.
json-c package - Everything work fine. But the report generated in blank. No license for binary.
I checked in scancode.json there is no entry for binary. Rest contents are OKAY!

On the other note, I have explored the source code of QuaterMaster. I came to below understanding,
QuaterMaster invokes ScanCode for licensing and ScanCode invokes strings to get the license from binary. Please note, I have also explored the Scancode.
So basically, what output ''$strings -a <binary_name> | grep license'' gives is the license for the binary.
Please correct if my understanding is wrong.
Do QuaterMaster gives the information from which source files this license has come into the binary?

Regards,

@GiasemiSh
Copy link

Scancode detects licenses from the source files.
Quartermaster creates a build graph, from the source files to the resulting package.
We add the license information we have collected from ScanCode to the source files and through the build graph we know which licenses come into the binary or the package as a whole.

Now if you want to view the results I would ask you to make a change in your yaml configuration.

--> In order to deploy this option you need to have Hugo installed in your machine.

In the qmstr-reporter-html configuration please add the following line:
generatehtml: "no"

So the qmstr-reporter-html configuration would look like this:

- reporter: qmstr-reporter-html
      name: "HTML Reporter"
      config:
        generatehtml: "no"
        siteprovider: "Endocode"
        baseurl: "http://localhost:8080/"

With this option, you disable qmstr to execute Hugo which generates the static HTML page with the QMSTR reports. Since it's disabled you can afterwards run it locally and view the results.

Please build the json-c package with QMSTR again.
After QMSTR has finished and quit, you can go to the qmstr-reporter-html output, execute Hugo locally and view the results:

> cd qmstr/qmstr-reporter-html/HTML_Reporter
> ls
qmstr-reports.tar.bz2

> tar -xf qmstr-reports.tar.bz2
> cd qmstr-your_random_id
> hugo server

You should be able to view the report results at http://localhost:1313/

@utkarshyen
Copy link
Author

Hello @GiasemiSh ,

Thank you for your detailed info. It was really helpful.
Also tried your config changes.
However, I could see the page on local host but data was not present. Even from hugo server I could not see any entries in the report.

I checked the scancode.json and it is fine.
PFA ss of the report.
Meanwhile investigating on my side.

Regards,

elfutils_report

@utkarshyen
Copy link
Author

Hello,

We add the license information we have collected from ScanCode to the source files and through the build graph we know which licenses come into the binary or the package as a whole.

I have seen some sample report of curl on https://qmstr.org/packages/curl/curl/curl-7_64_1-142-g761b49b91/ which gives the licenses for binary. But no info of source list used or which license belongs to which source files.
Am I missing anything here?

For e.g.
filea licensea
fileb licenseb
...
...
filen licensen

However, binary is built with only filea and thus licensea. Does QuaterMaster provides such information. I could answer these questions myself once I successfully setup and run the QuaterMaster my side and explore. Just sharing my view.

Regards,

@fullsushidev fullsushidev removed their assignment Aug 13, 2020
@fullsushidev fullsushidev added this to the Backlog milestone Aug 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants