Exporter/OcAgent: Add to release artifacts.

[songy23]

The only thing left with OCAgent Metrics exporter is #1637, which is a non-user-facing minor change.

Going to test the Java exporter against the real Agent before merging this PR.

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@d1ecb19). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #1761   +/-   ##
=========================================
  Coverage          ?   82.24%           
  Complexity        ?     1977           
=========================================
  Files             ?      291           
  Lines             ?     9179           
  Branches          ?      885           
=========================================
  Hits              ?     7549           
  Misses            ?     1336           
  Partials          ?      294

Impacted Files	Coverage Δ	Complexity Δ
...sus/exporter/metrics/ocagent/OcAgentNodeUtils.java	0% <ø> (ø)	0 <0> (?)
...s/ocagent/OcAgentMetricsExporterConfiguration.java	100% <ø> (ø)	2 <0> (?)
...porter/metrics/ocagent/OcAgentMetricsExporter.java	0% <ø> (ø)	0 <0> (?)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d1ecb19...42272c0. Read the comment docs.

[bogdandrutu]

Let me know how the test went.

[songy23]

I was able to use the Java OC-Agent Metrics Exporter to dial to Agent and then send metrics RPC requests. However on the Agent side those metrics didn't seem to arrive, though on the Exporter side everything was sent successfully without any errors. Some logs from Exporter:

...
log4j:WARN No appenders could be found for logger (io.grpc.netty.shaded.io.netty.util.internal.logging.InternalLoggerFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Feb 19, 2019 1:32:22 PM io.opencensus.exporter.metrics.ocagent.OcAgentMetricsServiceExportRpcHandler onExport
INFO: Sending request node {
  identifier {
    host_name: "songya.svl.corp.google.com"
    pid: 224556
    start_timestamp {
      seconds: 1550611942
      nanos: 490000000
    }
  }
  library_info {
    language: JAVA
    exporter_version: "0.18.0-SNAPSHOT"
    core_library_version: "0.20.0-SNAPSHOT"
  }
  service_info {
    name: "ocagent-java-example"
  }
}

Feb 19, 2019 1:32:22 PM io.opencensus.exporter.metrics.ocagent.OcAgentMetricsServiceExportRpcHandler onExport
INFO: Sending request metrics {
  metric_descriptor {
    name: "task_latency_distribution"
    description: "The distribution of the task latencies."
    unit: "ms"
    type: CUMULATIVE_DISTRIBUTION
  }
  timeseries {
    start_timestamp {
      seconds: 1550611939
      nanos: 880000000
    }
    points {
      timestamp {
        seconds: 1550611942
        nanos: 583000000
      }
      distribution_value {
        count: 100
        sum: 258037.9999999999
        sum_of_squared_deviation: 2.0919464155999994E8
        bucket_options {
          explicit {
            bounds: 100.0
            bounds: 200.0
            bounds: 400.0
            bounds: 1000.0
            bounds: 2000.0
            bounds: 4000.0
          }
        }
        buckets {
          count: 3
        }
        buckets {
          count: 1
        }
        buckets {
          count: 6
        }
        buckets {
          count: 10
        }
        buckets {
          count: 16
        }
        buckets {
          count: 41
        }
        buckets {
          count: 23
        }
      }
    }
  }
}
...

Full code example: https://gist.github.com/songy23/12c03c6d9db7955b94e0c626090477e1.

Could be something wrong with the Agent setup. Am working on debugging.

[songy23]

Update:

Did some interop tests and found interesting results. I tested with Go-based Exporter and Agent, as well as Java based Exporter and (fake) Agent:

• Go exporter to Go Agent: works perfectly fine.
• Go exporter to Java fake Agent: works perfectly fine.
• Java exporter to Java fake Agent: works perfectly fine.
• Java exporter to Go Agent: connection is established and messages are sent successfully from Exporter side, but somehow all messages are dropped before they even reached the receiver.

[bogdandrutu]

@songy23 did you check the zpages on the client and server to see if the spans are not closed with errors?

[songy23]

Apologies for not getting back to this earlier. I've been experimenting with the example in #1788, and the problem persists. Java exporters still worked fine with Java server, but it seems Go Agent cannot deserialize messages from Java exporters. There are no stats/spans on the zpages, no errors/logs either - it's like all the bytes from Java client are silently dropped. This is the case for both metrics and tracing ocagent exporters.

@odeke-em I might need your help on debugging this if you got time.

[odeke-em]

Apologies for not getting back to this earlier. I've been experimenting with the example in #1788, and the problem persists. Java exporters still worked fine with Java server, but it seems Go Agent cannot deserialize messages from Java exporters. There are no stats/spans on the zpages, no errors/logs either - it's like all the bytes from Java client are silently dropped. This is the case for both metrics and tracing ocagent exporters.

@odeke-em I might need your help on debugging this if you got time.

@songy23 sure, let's do it. Wanna have a call?

[odeke-em]

I highly suspect that this is just an issue with OpenCensus-Java using an old version of OpenCensus-Proto i.e. https://mvnrepository.com/artifact/io.opencensus/opencensus-proto/0.1.0 from November 2018

which has had at least 23 updates since that v0.1.0 release

yet the agent/service uses an updated version from mid-December 2018 so that we could have the grpc-gateway github.com/census-instrumentation/opencensus-proto v0.1.0-0.20181214143942-ba49f56771b8.
I've helped many folks debug the exact same issue and 2 months ago which is why I filed a request for us to make a publication of OpenCensus-Proto census-instrumentation/opencensus-proto#161 but that hasn't yet happened.

If you make a git checkout of the agent to coincide with those dates, this exporter should work and show you zPages actively receiving and exporting data.

[songy23]

@odeke-em You're right, the inconsistent opencensus-proto versions are the root cause. Many thanks for the suggestion!

After downgrading the opencensus-proto version in Agent to v0.1.0 and rebuilding it, the Java exporters were finally able to talk to Agent. TraceZ page:

I'll put this PR on hold until we made the new proto release and use the same version between OC-Java and OC-Service.

[songy23]

Blocked by census-instrumentation/opencensus-proto#161.

[songy23]

v0.2.0 release of opencensus-proto is available on Maven and opencensus-java has upgraded the version in #1795. OC-Agent will migrate to opencensus-proto v0.2.0 in census-instrumentation/opencensus-service#490. It should be safe to release the Java agent exporters at this point. Thanks for the review!