Skip to content

Test compatibility with other tools #17

Description

@irongut

Feature Request

CCS has been tested with Coverlet and gcov, test with other tools that output Cobertura format.

Expected Behaviour

Obtain sample Coberura output from as many testing tools as possible, CCS should parse them correctly.

Additional Context

Linked To

#20 What does complexity mean? (issue with coverage.py)
#22 Allow hiding some output values

Activity

  1. self-assigned this
    on Oct 16, 2021
  2. added
    ParsingAn issue reletad to parsing the coverage xml.
    on Oct 16, 2021
  3. irongut commented on Oct 29, 2021

    @irongut
    OwnerAuthor

    coverage.py Sample

    <coverage branch-rate="0" branches-covered="0" branches-valid="0" complexity="0" line-rate="0.6500" lines-covered="65" lines-valid="100" timestamp="1635484070186" version="6.0.2">
        <packages>
            <package branch-rate="0" complexity="0" line-rate="0.6500" name="one" />
            <package branch-rate="0" complexity="0" line-rate="0.6000" name="two" />
            <package branch-rate="0" complexity="0" line-rate="0.7000" name="three" />
        </packages>
    </coverage>

    (Details not relevant to CCS omitted.)

  4. pinned this issue on Oct 29, 2021
  5. pboling commented on Feb 17, 2022

    @pboling

    Tried it with simplecov & simplecov-cobertura, and it failed, but that may be caused by the options I have enabled not being supported by simplecov? If that's the case this error should be handled more gracefully or avoided altogether.

    Here's the log:

    Run irongut/CodeCoverageSummary@v1.[2](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:2).0
      with:
        filename: ./coverage/coverage.xml
        badge: true
        fail_below_min: true
        format: markdown
        hide_branch_rate: false
        hide_complexity: true
        indicators: true
        output: both
        thresholds: 95 97
      env:
        CC_TEST_REPORTER_ID: ***
        GIT_COMMIT_SHA: 5e7798ac8bfdc4f[3](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:3)d8537d[4](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:4)23af23be[5](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:5)f33ec[6](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:6)81
        GIT_BRANCH: 1-4-stable
    /usr/bin/docker run --name ghcrioirongutcodecoveragesummaryv120_db1[7](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:7)64 --label 7db2d7 --workdir /github/workspace --rm -e CC_TEST_REPORTER_ID -e GIT_COMMIT_SHA -e GIT_BRANCH -e INPUT_FILENAME -e INPUT_BADGE -e INPUT_FAIL_BELOW_MIN -e INPUT_FORMAT -e INPUT_HIDE_BRANCH_RATE -e INPUT_HIDE_COMPLEXITY -e INPUT_INDICATORS -e INPUT_OUTPUT -e INPUT_THRESHOLDS -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_REF_NAME -e GITHUB_REF_PROTECTED -e GITHUB_REF_TYPE -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_ARCH -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/oauth2/oauth2":"/github/workspace" ghcr.io/irongut/codecoveragesummary:v1.2.0  "--files" "./coverage/coverage.xml" "--badge" "true" "--fail" "true" "--format" "markdown" "--hidebranch" "false" "--hidecomplexity" "true" "--indicators" "true" "--output" "both" "--thresholds" "[9](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:9)5 97"
    Code Coverage File: ./coverage/coverage.xml
    Parse Error: Sequence contains no elements
    Error: System.NullReferenceException - Object reference not set to an instance of an object.
    

    Going to tweak options and see if I can get it working.

  6. pboling commented on Feb 17, 2022

    @pboling

    Simplecov Sample (Ruby, branch coverage not enabled)

    <?xml version='1.0'?> 
    <!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
    <!-- Generated by simplecov-cobertura version 2.1.0 (https://github.com/dashingrocket/simplecov-cobertura) -->
    <coverage line-rate="0.97" lines-covered="350" lines-valid="362" complexity="0" version="0" timestamp="1645060052">
      <sources>
        <source>/Users/pboling/src/my/oauth2</source>
      </sources>
      <packages>
        <package name="oauth2" line-rate="0.97" complexity="0">
          <classes>
            <class name="oauth2" filename="lib/oauth2.rb" line-rate="1.0" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="3" hits="1"/>
                <line number="4" hits="1"/>
                <line number="5" hits="1"/>
                <line number="6" hits="1"/>
                <line number="7" hits="1"/>
                <line number="8" hits="1"/>
                <line number="9" hits="1"/>
                <line number="10" hits="1"/>
                <line number="11" hits="1"/>
                <line number="12" hits="1"/>
              </lines>
            </class>
            <class name="access_token" filename="lib/oauth2/access_token.rb" line-rate="0.96" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="3" hits="1"/>
                <line number="4" hits="1"/>
                <line number="7" hits="1"/>
                <line number="13" hits="1"/>
                <line number="14" hits="7"/>
                <line number="15" hits="7"/>
                <line number="23" hits="1"/>
                <line number="24" hits="1"/>
                <line number="41" hits="1"/>
                <line number="42" hits="151"/>
                <line number="43" hits="151"/>
                <line number="44" hits="151"/>
                <line number="45" hits="151"/>
                <line number="46" hits="453"/>
                <line number="48" hits="151"/>
                <line number="49" hits="151"/>
                <line number="50" hits="151"/>
                <line number="51" hits="151"/>
                <line number="52" hits="151"/>
                <line number="55" hits="151"/>
                <line number="61" hits="1"/>
                <line number="62" hits="6"/>
                <line number="68" hits="1"/>
                <line number="69" hits="11"/>
                <line number="75" hits="1"/>
                <line number="76" hits="3"/>
                <line number="83" hits="1"/>
                <line number="84" hits="2"/>
                <line number="86" hits="2"/>
                <line number="87" hits="2"/>
                <line number="88" hits="2"/>
                <line number="89" hits="2"/>
                <line number="90" hits="2"/>
                <line number="91" hits="2"/>
                <line number="97" hits="1"/>
                <line number="98" hits="1"/>
                <line number="107" hits="1"/>
                <line number="108" hits="20"/>
                <line number="109" hits="20"/>
                <line number="115" hits="1"/>
                <line number="116" hits="2"/>
                <line number="122" hits="1"/>
                <line number="123" hits="18"/>
                <line number="129" hits="1"/>
                <line number="130" hits="2"/>
                <line number="136" hits="1"/>
                <line number="137" hits="0"/>
                <line number="143" hits="1"/>
                <line number="144" hits="2"/>
                <line number="148" hits="1"/>
                <line number="149" hits="8"/>
                <line number="152" hits="1"/>
                <line number="154" hits="1"/>
                <line number="155" hits="20"/>
                <line number="157" hits="8"/>
                <line number="158" hits="8"/>
                <line number="160" hits="8"/>
                <line number="161" hits="8"/>
                <line number="163" hits="4"/>
                <line number="164" hits="4"/>
                <line number="165" hits="4"/>
                <line number="167" hits="0"/>
                <line number="171" hits="0"/>
                <line number="175" hits="1"/>
                <line number="176" hits="13"/>
                <line number="178" hits="12"/>
              </lines>
            </class>
            <class name="authenticator" filename="lib/oauth2/authenticator.rb" line-rate="1.0" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="3" hits="1"/>
                <line number="4" hits="1"/>
                <line number="5" hits="1"/>
                <line number="7" hits="1"/>
                <line number="8" hits="106"/>
                <line number="9" hits="106"/>
                <line number="10" hits="106"/>
                <line number="22" hits="1"/>
                <line number="23" hits="106"/>
                <line number="25" hits="14"/>
                <line number="27" hits="89"/>
                <line number="29" hits="1"/>
                <line number="31" hits="1"/>
                <line number="33" hits="1"/>
                <line number="37" hits="1"/>
                <line number="38" hits="15"/>
                <line number="41" hits="1"/>
                <line number="45" hits="1"/>
                <line number="46" hits="89"/>
                <line number="51" hits="1"/>
                <line number="52" hits="1"/>
                <line number="57" hits="1"/>
                <line number="58" hits="14"/>
                <line number="59" hits="14"/>
                <line number="60" hits="14"/>
                <line number="64" hits="1"/>
                <line number="65" hits="14"/>
              </lines>
            </class>
            <class name="client" filename="lib/oauth2/client.rb" line-rate="0.95" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="4" hits="1"/>
                <line number="6" hits="1"/>
                <line number="7" hits="1"/>
                <line number="9" hits="1"/>
                <line number="10" hits="1"/>
                <line number="11" hits="1"/>
                <line number="31" hits="1"/>
                <line number="32" hits="204"/>
                <line number="33" hits="204"/>
                <line number="34" hits="204"/>
                <line number="35" hits="204"/>
                <line number="36" hits="204"/>
                <line number="39" hits="204"/>
                <line number="49" hits="204"/>
                <line number="55" hits="1"/>
                <line number="56" hits="0"/>
                <line number="57" hits="0"/>
                <line number="61" hits="1"/>
                <line number="62" hits="423"/>
                <line number="63" hits="158"/>
                <line number="64" hits="158"/>
                <line number="65" hits="134"/>
                <line number="66" hits="134"/>
                <line number="69" hits="158"/>
                <line number="76" hits="1"/>
                <line number="77" hits="11"/>
                <line number="78" hits="11"/>
                <line number="84" hits="1"/>
                <line number="85" hits="100"/>
                <line number="100" hits="1"/>
                <line number="101" hits="141"/>
                <line number="103" hits="141"/>
                <line number="105" hits="141"/>
                <line number="106" hits="141"/>
                <line number="107" hits="141"/>
                <line number="109" hits="141"/>
                <line number="111" hits="141"/>
                <line number="113" hits="3"/>
                <line number="114" hits="3"/>
                <line number="115" hits="3"/>
                <line number="117" hits="2"/>
                <line number="118" hits="1"/>
                <line number="119" hits="1"/>
                <line number="121" hits="2"/>
                <line number="124" hits="127"/>
                <line number="126" hits="11"/>
                <line number="127" hits="11"/>
                <line number="129" hits="2"/>
                <line number="130" hits="2"/>
                <line number="132" hits="0"/>
                <line number="133" hits="0"/>
                <line number="143" hits="1"/>
                <line number="144" hits="97"/>
                <line number="145" hits="182"/>
                <line number="146" hits="2"/>
                <line number="148" hits="180"/>
                <line number="151" hits="97"/>
                <line number="153" hits="97"/>
                <line number="154" hits="97"/>
                <line number="155" hits="97"/>
                <line number="156" hits="97"/>
                <line number="157" hits="79"/>
                <line number="158" hits="79"/>
                <line number="160" hits="18"/>
                <line number="161" hits="18"/>
                <line number="163" hits="97"/>
                <line number="164" hits="97"/>
                <line number="167" hits="97"/>
                <line number="169" hits="0"/>
                <line number="172" hits="97"/>
                <line number="173" hits="3"/>
                <line number="174" hits="3"/>
                <line number="176" hits="94"/>
                <line number="182" hits="1"/>
                <line number="183" hits="48"/>
                <line number="189" hits="1"/>
                <line number="190" hits="5"/>
                <line number="196" hits="1"/>
                <line number="197" hits="11"/>
                <line number="203" hits="1"/>
                <line number="204" hits="21"/>
                <line number="207" hits="1"/>
                <line number="208" hits="9"/>
                <line number="227" hits="1"/>
                <line number="228" hits="55"/>
                <line number="229" hits="2"/>
                <line number="231" hits="53"/>
                <line number="235" hits="1"/>
                <line number="236" hits="91"/>
                <line number="237" hits="91"/>
                <line number="240" hits="1"/>
                <line number="242" hits="1"/>
                <line number="243" hits="97"/>
                <line number="244" hits="97"/>
                <line number="246" hits="93"/>
                <line number="250" hits="93"/>
                <line number="251" hits="1"/>
                <line number="253" hits="92"/>
              </lines>
            </class>
            <class name="error" filename="lib/oauth2/error.rb" line-rate="1.0" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="3" hits="1"/>
                <line number="7" hits="1"/>
                <line number="8" hits="14"/>
                <line number="9" hits="14"/>
                <line number="11" hits="14"/>
                <line number="12" hits="7"/>
                <line number="13" hits="7"/>
                <line number="14" hits="7"/>
                <line number="17" hits="14"/>
                <line number="23" hits="1"/>
                <line number="24" hits="14"/>
                <line number="26" hits="14"/>
                <line number="28" hits="14"/>
                <line number="29" hits="7"/>
                <line number="30" hits="7"/>
                <line number="32" hits="7"/>
                <line number="35" hits="14"/>
                <line number="37" hits="14"/>
              </lines>
            </class>
            <class name="mac_token" filename="lib/oauth2/mac_token.rb" line-rate="0.95" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="3" hits="1"/>
                <line number="4" hits="1"/>
                <line number="6" hits="1"/>
                <line number="7" hits="1"/>
                <line number="14" hits="1"/>
                <line number="15" hits="3"/>
                <line number="18" hits="1"/>
                <line number="30" hits="1"/>
                <line number="31" hits="19"/>
                <line number="32" hits="19"/>
                <line number="34" hits="18"/>
                <line number="43" hits="1"/>
                <line number="44" hits="4"/>
                <line number="46" hits="4"/>
                <line number="47" hits="4"/>
                <line number="49" hits="4"/>
                <line number="53" hits="1"/>
                <line number="54" hits="1"/>
                <line number="61" hits="1"/>
                <line number="62" hits="9"/>
                <line number="63" hits="9"/>
                <line number="65" hits="9"/>
                <line number="67" hits="8"/>
                <line number="69" hits="7"/>
                <line number="71" hits="7"/>
                <line number="80" hits="1"/>
                <line number="82" hits="8"/>
                <line number="91" hits="8"/>
                <line number="97" hits="1"/>
                <line number="98" hits="19"/>
                <line number="101" hits="1"/>
                <line number="103" hits="0"/>
                <line number="107" hits="17"/>
                <line number="109" hits="0"/>
                <line number="112" hits="1"/>
                <line number="116" hits="1"/>
                <line number="120" hits="1"/>
                <line number="124" hits="1"/>
                <line number="125" hits="8"/>
              </lines>
            </class>
            <class name="response" filename="lib/oauth2/response.rb" line-rate="1.0" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="3" hits="1"/>
                <line number="5" hits="1"/>
                <line number="7" hits="1"/>
                <line number="8" hits="1"/>
                <line number="9" hits="1"/>
                <line number="14" hits="62"/>
                <line number="15" hits="44"/>
                <line number="16" hits="3"/>
                <line number="20" hits="1"/>
                <line number="32" hits="1"/>
                <line number="33" hits="3"/>
                <line number="34" hits="3"/>
                <line number="35" hits="3"/>
                <line number="36" hits="6"/>
                <line number="46" hits="1"/>
                <line number="47" hits="147"/>
                <line number="48" hits="147"/>
                <line number="52" hits="1"/>
                <line number="53" hits="6"/>
                <line number="57" hits="1"/>
                <line number="58" hits="150"/>
                <line number="62" hits="1"/>
                <line number="63" hits="154"/>
                <line number="69" hits="1"/>
                <line number="70" hits="134"/>
                <line number="72" hits="131"/>
                <line number="76" hits="1"/>
                <line number="77" hits="244"/>
                <line number="81" hits="1"/>
                <line number="82" hits="244"/>
                <line number="84" hits="244"/>
                <line number="89" hits="1"/>
                <line number="90" hits="1"/>
              </lines>
            </class>
            <class name="assertion" filename="lib/oauth2/strategy/assertion.rb" line-rate="0.89" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="3" hits="1"/>
                <line number="4" hits="1"/>
                <line number="23" hits="1"/>
                <line number="27" hits="1"/>
                <line number="28" hits="1"/>
                <line number="45" hits="1"/>
                <line number="46" hits="8"/>
                <line number="47" hits="8"/>
                <line number="50" hits="1"/>
                <line number="51" hits="8"/>
                <line number="53" hits="8"/>
                <line number="60" hits="1"/>
                <line number="62" hits="8"/>
                <line number="67" hits="8"/>
                <line number="68" hits="8"/>
                <line number="69" hits="0"/>
                <line number="70" hits="0"/>
              </lines>
            </class>
            <class name="auth_code" filename="lib/oauth2/strategy/auth_code.rb" line-rate="1.0" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="6" hits="1"/>
                <line number="10" hits="1"/>
                <line number="11" hits="3"/>
                <line number="17" hits="1"/>
                <line number="18" hits="3"/>
                <line number="27" hits="1"/>
                <line number="28" hits="44"/>
                <line number="30" hits="44"/>
              </lines>
            </class>
            <class name="base" filename="lib/oauth2/strategy/base.rb" line-rate="1.0" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="3" hits="1"/>
                <line number="4" hits="1"/>
                <line number="5" hits="95"/>
              </lines>
            </class>
            <class name="client_credentials" filename="lib/oauth2/strategy/client_credentials.rb" line-rate="1.0" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="6" hits="1"/>
                <line number="10" hits="1"/>
                <line number="11" hits="1"/>
                <line number="18" hits="1"/>
                <line number="19" hits="20"/>
                <line number="20" hits="20"/>
              </lines>
            </class>
            <class name="implicit" filename="lib/oauth2/strategy/implicit.rb" line-rate="1.0" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="6" hits="1"/>
                <line number="10" hits="1"/>
                <line number="11" hits="3"/>
                <line number="17" hits="1"/>
                <line number="18" hits="3"/>
                <line number="24" hits="1"/>
                <line number="25" hits="1"/>
              </lines>
            </class>
            <class name="password" filename="lib/oauth2/strategy/password.rb" line-rate="1.0" complexity="0">
              <methods/>
              <lines>
                <line number="1" hits="1"/>
                <line number="2" hits="1"/>
                <line number="6" hits="1"/>
                <line number="10" hits="1"/>
                <line number="11" hits="1"/>
                <line number="19" hits="1"/>
                <line number="20" hits="10"/>
                <line number="23" hits="10"/>
              </lines>
            </class>
          </classes>
        </package>
      </packages>
    </coverage>
    
  7. irongut commented on Feb 17, 2022

    @irongut
    OwnerAuthor

    Hi @pboling thanks for the sample, that's always useful.

    Looking at the sample there are no branch metrics on coverage or package nodes. Unfortunately the hide_branch_rate option only hides it from the report, the action still attempts to calculate branch metrics.

    It's funny how many different ways different tools can output what is supposed to be the same format. I'll need to make some changes to prevent attempting to calculate metrics that don't exist in the data.

  8. pboling commented on Feb 17, 2022

    @pboling

    @irongut Branch coverage can be turned on, but I've never tried it. I'll see what that generates now. But, yeah, definitely good to support the lack of any specific feature. Always assume the null case :)

  9. irongut commented on Feb 17, 2022

    @irongut
    OwnerAuthor

    @pboling Looking at the docs, enable branch coverage but leave primary coverage set to the default of line. That should be compatible with the current version of CCS.

    I'll add a note to the CCS docs and include a fix for missing branch rate in the next version.

  10. github-actions commented on May 28, 2022

    @github-actions

    This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days.

  11. github-actions commented on Jun 27, 2022

    @github-actions

    This issue was closed because it has been stale for 30 days with no activity.

  12. pboling commented on Jun 27, 2022

    @pboling

    NOTE: the suggested config is working. I wrote up a how to on dev.to.

    Screen Shot 2022-06-28 at 5 28 32 AM

  13. github-actions commented on Oct 23, 2022

    @github-actions

    This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days.

  14. github-actions commented on Nov 23, 2022

    @github-actions

    This issue was closed because it has been stale for 30 days with no activity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ParsingAn issue reletad to parsing the coverage xml.enhancementNew feature or requesthelp wantedExtra attention is neededstale

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions