Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use SPI to manage components #1412

Open
wants to merge 36 commits into
base: 2.x
Choose a base branch
from

Conversation

brettmc
Copy link
Collaborator

@brettmc brettmc commented Oct 18, 2024

To resolve our long-standing race conditions stemming from using composer's autoload->files to register SDK components at runtime, this changes things so that:

  • components can be registered in various composer.json files, under extra.spi
  • existing _register.php files manually register through SPI, but will be removed from autoload_files if SPI is allowed to run
  • SDK fetches factories from SPI instead of maintaining static arrays

If we move ahead with this approach, a follow-up PR could remove our various late-binding providers.

Copy link

netlify bot commented Oct 18, 2024

Deploy Preview for opentelemetry-php canceled.

Name Link
🔨 Latest commit 5ffe83b
🔍 Latest deploy log https://app.netlify.com/sites/opentelemetry-php/deploys/671f1d38fe1753000839d348

Copy link

codecov bot commented Oct 18, 2024

Codecov Report

Attention: Patch coverage is 31.16279% with 148 lines in your changes missing coverage. Please review.

Project coverage is 71.76%. Comparing base (a19455c) to head (e33845b).

Files with missing lines Patch % Lines
src/SDK/Common/Services/Loader.php 80.64% 12 Missing ⚠️
src/Contrib/Otlp/_register.php 0.00% 6 Missing ⚠️
...tension/Propagator/B3/B3MultiPropagatorFactory.php 0.00% 6 Missing ⚠️
...rc/Extension/Propagator/B3/B3PropagatorFactory.php 0.00% 6 Missing ⚠️
...r/CloudTrace/CloudTraceOneWayPropagatorFactory.php 0.00% 6 Missing ⚠️
...pagator/CloudTrace/CloudTracePropagatorFactory.php 0.00% 6 Missing ⚠️
...opagator/Jaeger/JaegerBaggagePropagatorFactory.php 0.00% 6 Missing ⚠️
...sion/Propagator/Jaeger/JaegerPropagatorFactory.php 0.00% 6 Missing ⚠️
src/Contrib/Grpc/GrpcTransportFactory.php 0.00% 4 Missing ⚠️
src/Contrib/Otlp/LogsExporterFactory.php 20.00% 4 Missing ⚠️
... and 29 more
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##                2.x    #1412      +/-   ##
============================================
- Coverage     72.31%   71.76%   -0.55%     
- Complexity     2729     2760      +31     
============================================
  Files           401      405       +4     
  Lines          8148     8186      +38     
============================================
- Hits           5892     5875      -17     
- Misses         2256     2311      +55     
Flag Coverage Δ
8.2 71.61% <31.16%> (-0.52%) ⬇️
8.3 71.60% <31.16%> (-0.53%) ⬇️
8.4 71.62% <31.16%> (-0.66%) ⬇️
8.5 71.56% <31.16%> (-0.63%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/SDK/Common/Export/Http/PsrUtils.php 92.64% <100.00%> (ø)
src/SDK/Logs/ExporterFactory.php 100.00% <100.00%> (ø)
src/SDK/Metrics/MeterProviderFactory.php 76.92% <100.00%> (ø)
src/SDK/Propagation/PropagatorFactory.php 95.00% <100.00%> (+0.88%) ⬆️
src/SDK/Resource/ResourceInfoFactory.php 97.67% <100.00%> (ø)
src/SDK/Trace/ExporterFactory.php 100.00% <100.00%> (ø)
...omponentProvider/Logs/LogRecordExporterConsole.php 0.00% <0.00%> (ø)
...K/ComponentProvider/Logs/LogRecordExporterOtlp.php 0.00% <0.00%> (ø)
...K/ComponentProvider/Metrics/MetricExporterOtlp.php 0.00% <0.00%> (ø)
...DK/ComponentProvider/Trace/SpanExporterConsole.php 0.00% <0.00%> (ø)
... and 35 more

... and 13 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a19455c...e33845b. Read the comment docs.

@brettmc brettmc changed the title generate registry from composer autoload-dump ptototype: generate registry from composer autoload-dump Oct 20, 2024
@brettmc brettmc changed the title ptototype: generate registry from composer autoload-dump prototype: generate registry from composer autoload-dump Oct 20, 2024
To resolve our long-standing race conditions stemming from using composer's autoload->files
to registry SDK components at runtime, this changes things so that:
- components are registed in various composer.json files, under the extra.opentelemetry.registry key
- a composer script is registered to write this config out to a JSON file
- the SDK Registry is modified to make manually registering components a no-op (currently behind a flag, OTEL_PHP_EXPERIMENTAL_JSON_REGISTRY),
  and instead configure itself from the generated JSON file

If we move ahead with this approach, a follow-up PR could tidy up the Registry and remove our various late-binding providers.
@brettmc brettmc force-pushed the composer-extra-registry branch from ffb28df to abd9944 Compare October 21, 2024 00:48
@Nevay
Copy link
Contributor

Nevay commented Oct 21, 2024

An alternative idea if we want to avoid duplicating the "load implementations from composer.json" approach:

The main difference between the solution in this PR and an SPI-based approach is the addition of a name/key for each implementation within the composer.json configuration. This information could instead be added as additional method in the interfaces1 to allow loading them using SPI.

Registry is currently used for two different types of implementations:

  • factories used to initialize the SDK from environment variables
  • transport factory implementations, which are also used outside of environment variables configuration

Transport factory implementations

We can extend TransportFactoryInterface with the necessary information to populate the transport factories:

$factories = iterator_to_array(ServiceLoader::load(TransportFactoryServiceInterface::class));
array_multisort(
    array_map(static fn($factory) => $factory->priority(), $factories),
    SORT_DESC,
    $factories,
);
$factoriesByProtocol = [];
foreach ($factories as $factory) {
    $factoriesByProtocol[$factory->protocol()] ??= $factory;
}

self::$transportFactories = $factoriesByProtocol;
interface TransportFactoryServiceInterface extends TransportFactoryInterface {

    public function protocol(): string;
    
    public function priority(): int;
}

Factories to initialize the SDK from environment variables

We could align the implementation with the file-based implementation by adding a new, from the current implementation independent interface2 similar to Configuration\ComponentProvider and deprecate the current registry methods and factory interfaces.
Very basic interface definition:

namespace OpenTelemetry\Config\SDK\Environment;

/**
 * @template T
 */
interface ComponentProvider {

    /**
     * @return T
     */
    public function createPlugin(): mixed;

    public function name(): string;
}

Alternatively we could follow the approach mentioned for transport factory implementations and add a name(/priority) method to the factory interfaces and continue using the Registry.

Footnotes

  1. See also Java ServiceLoader - Designing Services:

    A service should declare as many methods as needed to allow service providers to communicate their domain-specific properties and other quality-of-implementation factors. An application which obtains a service loader for the service may then invoke these methods on each instance of a service provider, in order to choose the best provider for the application.

  2. FWIW I've been using the following Env\Loader interface for env-based configuration, its implementations are loaded via SPI:

    /**
     * @template T
     */
    interface Loader {
    
        /**
         * @return T
         */
        public function load(EnvResolver $env, LoaderRegistry $registry, Context $context): mixed;
    
        public function name(): string;
    }
    

@brettmc
Copy link
Collaborator Author

brettmc commented Oct 22, 2024

updated to use SPI, and implemented the idea of having factories declare their key & priority.

@haad
Copy link

haad commented Oct 23, 2024

  • The Vneno i

Copy link
Contributor

@ChrisLightfootWild ChrisLightfootWild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an exciting set of changes!

Just left a few bits for consideration/discussion.

Copy link
Contributor

@xvilo xvilo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small code-style comment was placed.

I understand that the Registry stuff will not be used when using extra.spi. Is it correct that third-party libraries (in my case, proprietary code/packages within a company) can also add their instrumentation through the SPI way, as shown in composer.json?

It would be nice to pair this with some documentation regardless, e.g. a 'How to add instrumentation into your composer package' section. It would be a good way to demo/reflect/document the preferred way of including AutoInstrumentations.

src/Extension/Propagator/B3/B3MultiPropagatorFactory.php Outdated Show resolved Hide resolved
@brettmc
Copy link
Collaborator Author

brettmc commented Oct 23, 2024

I understand that the Registry stuff will not be used when using extra.spi. Is it correct that third-party libraries (in my case, proprietary code/packages within a company) can also add their instrumentation through the SPI way, as shown in composer.json?

@xvilo That's right. It's still up for debate exactly how registry/SPI should interact, and which one would be the default. The SPI mechanism will definitely allow custom components like we have now.

@brettmc brettmc added the breaking A breaking change label Jan 7, 2025
To resolve our long-standing race conditions stemming from using composer's autoload->files
to registry SDK components at runtime, this changes things so that:
- components are registed in various composer.json files, under the extra.opentelemetry.registry key
- a composer script is registered to write this config out to a JSON file
- the SDK Registry is modified to make manually registering components a no-op (currently behind a flag, OTEL_PHP_EXPERIMENTAL_JSON_REGISTRY),
  and instead configure itself from the generated JSON file

If we move ahead with this approach, a follow-up PR could tidy up the Registry and remove our various late-binding providers.
brettmc and others added 10 commits January 9, 2025 15:55
- conditionally use SPI in extensions _register.php
- deprecate OtlpHttpTransportFactory in favour of PsrTransportFactory
add the php-cs-fixer no_blank_lines_after_class_opening rule and apply to codebase
the spec has added in-development otlp file/stdout exporter. We already supported this programmatically, so
add some factories and sdk config to allow configuration from environment (only for stdout, per spec)
The event logger was a Development-status component of the logging signal. It has been
removed in favour of adding emitEvent to the logger interface,
see open-telemetry/opentelemetry-specification#4319
@brettmc brettmc changed the base branch from main to 2.x January 9, 2025 04:59
@brettmc brettmc changed the title prototype: generate registry from composer autoload-dump use SPI to manage components Jan 10, 2025
@brettmc brettmc force-pushed the composer-extra-registry branch from 934939d to e33845b Compare January 10, 2025 02:27
@brettmc brettmc marked this pull request as ready for review January 10, 2025 02:32
@brettmc brettmc requested a review from a team as a code owner January 10, 2025 02:32
@brettmc
Copy link
Collaborator Author

brettmc commented Jan 10, 2025

@open-telemetry/php-approvers @Nevay I think this is ready for review: NB that it targets 2.x branch and has some BC breakage, which I've started to document in docs/upgrading.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking A breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants