Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: pass parameters natively via http interface along the query #224

Merged
merged 1 commit into from
Jan 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Naming used here is the same as in ClickHouse docs.
- Works with any HTTP Client implementation ([PSR-18 compliant](https://www.php-fig.org/psr/psr-18/))
- All [ClickHouse Formats](https://clickhouse.yandex/docs/en/interfaces/formats/) support
- Logging ([PSR-3 compliant](https://www.php-fig.org/psr/psr-3/))
- SQL Factory for [parameters "binding"](#parameters-binding)
- [Native query parameters](#native-query-parameters) support

## Contents

Expand All @@ -29,7 +29,7 @@ Naming used here is the same as in ClickHouse docs.
- [Insert](#insert)
- [Async API](#async-api)
- [Select](#select-1)
- [Parameters "binding"](#parameters-binding)
- [Native Query Parameters](#native-query-parameters)
- [Snippets](#snippets)

## Setup
Expand Down Expand Up @@ -227,7 +227,10 @@ If not provided they're not passed either:

### Select

## Parameters "binding"
## Native Query Parameters

> [!TIP]
> [Official docs](https://clickhouse.com/docs/en/interfaces/http#cli-queries-with-parameters)

```php
<?php
Expand All @@ -238,17 +241,14 @@ use SimPod\ClickHouseClient\Sql\ValueFormatter;
$sqlFactory = new SqlFactory(new ValueFormatter());

$sql = $sqlFactory->createWithParameters(
'SELECT :param',
'SELECT {p1:String}',
['param' => 'value']
);
```
This produces `SELECT 'value'` and it can be passed to `ClickHouseClient::select()`.
This produces `SELECT 'value'` in ClickHouse and it can be passed to `ClickHouseClient::select()`.

Supported types are:
- scalars
- DateTimeImmutable (`\DateTime` is not supported because `ValueFormatter` might modify its timezone so it's not considered safe)
- [Expression](#expression)
- objects implementing `__toString()`
All types are supported (except `AggregateFunction`, `SimpleAggregateFunction` and `Nothing` by design).
You can also pass `DateTimeInterface` into `Date*` types or native array into `Array`, `Tuple`, `Native` and `Geo` types

### Expression

Expand Down
9 changes: 6 additions & 3 deletions src/Client/ClickHouseClient.php
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
use Psr\Http\Client\ClientExceptionInterface;
use SimPod\ClickHouseClient\Exception\CannotInsert;
use SimPod\ClickHouseClient\Exception\ServerError;
use SimPod\ClickHouseClient\Exception\UnsupportedValue;
use SimPod\ClickHouseClient\Exception\UnsupportedParamType;
use SimPod\ClickHouseClient\Exception\UnsupportedParamValue;
use SimPod\ClickHouseClient\Format\Format;
use SimPod\ClickHouseClient\Output\Output;

Expand All @@ -27,7 +28,8 @@ public function executeQuery(string $query, array $settings = []): void;
*
* @throws ClientExceptionInterface
* @throws ServerError
* @throws UnsupportedValue
* @throws UnsupportedParamType
* @throws UnsupportedParamValue
*/
public function executeQueryWithParams(string $query, array $params, array $settings = []): void;

Expand All @@ -53,7 +55,8 @@ public function select(string $query, Format $outputFormat, array $settings = []
*
* @throws ClientExceptionInterface
* @throws ServerError
* @throws UnsupportedValue
* @throws UnsupportedParamType
* @throws UnsupportedParamValue
*
* @template O of Output
*/
Expand Down
31 changes: 31 additions & 0 deletions src/Client/Http/RequestFactory.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,15 @@
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\UriFactoryInterface;
use Psr\Http\Message\UriInterface;
use SimPod\ClickHouseClient\Exception\UnsupportedParamType;
use SimPod\ClickHouseClient\Param\ParamValueConverterRegistry;
use SimPod\ClickHouseClient\Sql\Type;

use function array_keys;
use function array_reduce;
use function http_build_query;
use function is_string;
use function preg_match_all;
use function SimPod\ClickHouseClient\absurd;

use const PHP_QUERY_RFC3986;
Expand All @@ -23,6 +29,7 @@

/** @throws InvalidArgumentException */
public function __construct(
private ParamValueConverterRegistry $paramValueConverterRegistry,
private RequestFactoryInterface $requestFactory,
UriFactoryInterface|null $uriFactory = null,
UriInterface|string $uri = '',
Expand All @@ -40,6 +47,7 @@
$this->uri = $uri;
}

/** @throws UnsupportedParamType */
public function prepareRequest(RequestOptions $requestOptions): RequestInterface
{
$query = http_build_query(
Expand All @@ -62,7 +70,30 @@

$request = $this->requestFactory->createRequest('POST', $uri);

preg_match_all('~\{([a-zA-Z\d]+):([a-zA-Z\d ]+(\(.+\))?)}~', $requestOptions->sql, $matches);

$typeToParam = array_reduce(
array_keys($matches[1]),
static function (array $acc, string|int $k) use ($matches) {
$acc[$matches[1][$k]] = Type::fromString($matches[2][$k]);

return $acc;
},
[],
);

$streamElements = [['name' => 'query', 'contents' => $requestOptions->sql]];
foreach ($requestOptions->params as $name => $value) {
$type = $typeToParam[$name] ?? null;
if ($type === null) {
continue;

Check warning on line 89 in src/Client/Http/RequestFactory.php

View check run for this annotation

Codecov / codecov/patch

src/Client/Http/RequestFactory.php#L89

Added line #L89 was not covered by tests
}

$streamElements[] = [
'name' => 'param_' . $name,
'contents' => $this->paramValueConverterRegistry->get($type)($value, $type, false),
];
}

try {
$body = new MultipartStream($streamElements);
Expand Down
9 changes: 7 additions & 2 deletions src/Client/Http/RequestOptions.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,16 @@ final class RequestOptions
public array $settings;

/**
* @param array<string, mixed> $params
* @param array<string, float|int|string> $defaultSettings
* @param array<string, float|int|string> $querySettings
*/
public function __construct(public string $sql, array $defaultSettings, array $querySettings)
{
public function __construct(
public string $sql,
public array $params,
array $defaultSettings,
array $querySettings,
) {
$this->settings = $querySettings + $defaultSettings;
}
}
10 changes: 8 additions & 2 deletions src/Client/PsrClickHouseAsyncClient.php
Original file line number Diff line number Diff line change
Expand Up @@ -62,25 +62,31 @@ public function selectWithParams(
$sql
$formatClause
CLICKHOUSE,
$settings,
static fn (ResponseInterface $response): Output => $outputFormat::output($response->getBody()->__toString())
params: $params,
settings: $settings,
processResponse: static fn (ResponseInterface $response): Output => $outputFormat::output(
$response->getBody()->__toString(),
)
);
}

/**
* @param array<string, mixed> $params
* @param array<string, float|int|string> $settings
* @param (callable(ResponseInterface):mixed)|null $processResponse
*
* @throws Exception
*/
private function executeRequest(
string $sql,
array $params,
array $settings = [],
callable|null $processResponse = null,
): PromiseInterface {
$request = $this->requestFactory->prepareRequest(
new RequestOptions(
$sql,
$params,
$this->defaultSettings,
$settings,
),
Expand Down
56 changes: 38 additions & 18 deletions src/Client/PsrClickHouseClient.php
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@
use SimPod\ClickHouseClient\Client\Http\RequestOptions;
use SimPod\ClickHouseClient\Exception\CannotInsert;
use SimPod\ClickHouseClient\Exception\ServerError;
use SimPod\ClickHouseClient\Exception\UnsupportedValue;
use SimPod\ClickHouseClient\Exception\UnsupportedParamType;
use SimPod\ClickHouseClient\Exception\UnsupportedParamValue;
use SimPod\ClickHouseClient\Format\Format;
use SimPod\ClickHouseClient\Output\Output;
use SimPod\ClickHouseClient\Sql\Escaper;
Expand Down Expand Up @@ -46,13 +47,18 @@

public function executeQuery(string $query, array $settings = []): void
{
$this->executeRequest($query, settings: $settings);
try {
$this->executeRequest($query, params: [], settings: $settings);
} catch (UnsupportedParamType) {
absurd();

Check warning on line 53 in src/Client/PsrClickHouseClient.php

View check run for this annotation

Codecov / codecov/patch

src/Client/PsrClickHouseClient.php#L52-L53

Added lines #L52 - L53 were not covered by tests
}
}

public function executeQueryWithParams(string $query, array $params, array $settings = []): void
{
$this->executeRequest(
$this->sqlFactory->createWithParameters($query, $params),
params: $params,
settings: $settings,
);
}
Expand All @@ -61,7 +67,7 @@
{
try {
return $this->selectWithParams($query, params: [], outputFormat: $outputFormat, settings: $settings);
} catch (UnsupportedValue) {
} catch (UnsupportedParamValue | UnsupportedParamType) {
absurd();
}
}
Expand All @@ -77,6 +83,7 @@
$sql
$formatClause
CLICKHOUSE,
params: $params,
settings: $settings,
);

Expand Down Expand Up @@ -112,14 +119,19 @@

$table = Escaper::quoteIdentifier($table);

$this->executeRequest(
<<<CLICKHOUSE
INSERT INTO $table
$columnsSql
VALUES $valuesSql
CLICKHOUSE,
settings: $settings,
);
try {
$this->executeRequest(
<<<CLICKHOUSE
INSERT INTO $table
$columnsSql
VALUES $valuesSql
CLICKHOUSE,
params: [],
settings: $settings,
);
} catch (UnsupportedParamType) {
absurd();

Check warning on line 133 in src/Client/PsrClickHouseClient.php

View check run for this annotation

Codecov / codecov/patch

src/Client/PsrClickHouseClient.php#L133

Added line #L133 was not covered by tests
}
}

public function insertWithFormat(string $table, Format $inputFormat, string $data, array $settings = []): void
Expand All @@ -128,25 +140,33 @@

$table = Escaper::quoteIdentifier($table);

$this->executeRequest(
<<<CLICKHOUSE
INSERT INTO $table $formatSql $data
CLICKHOUSE,
settings: $settings,
);
try {
$this->executeRequest(
<<<CLICKHOUSE
INSERT INTO $table $formatSql $data
CLICKHOUSE,
params: [],
settings: $settings,
);
} catch (UnsupportedParamType) {
absurd();

Check warning on line 152 in src/Client/PsrClickHouseClient.php

View check run for this annotation

Codecov / codecov/patch

src/Client/PsrClickHouseClient.php#L151-L152

Added lines #L151 - L152 were not covered by tests
}
}

/**
* @param array<string, mixed> $params
* @param array<string, float|int|string> $settings
*
* @throws ServerError
* @throws ClientExceptionInterface
* @throws UnsupportedParamType
*/
private function executeRequest(string $sql, array $settings): ResponseInterface
private function executeRequest(string $sql, array $params, array $settings): ResponseInterface
{
$request = $this->requestFactory->prepareRequest(
new RequestOptions(
$sql,
$params,
$this->defaultSettings,
$settings,
),
Expand Down
21 changes: 21 additions & 0 deletions src/Exception/UnsupportedParamType.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<?php

declare(strict_types=1);

namespace SimPod\ClickHouseClient\Exception;

use InvalidArgumentException;
use SimPod\ClickHouseClient\Sql\Type;

final class UnsupportedParamType extends InvalidArgumentException implements ClickHouseClientException
{
public static function fromType(Type $type): self
{
return new self($type->name);
}

public static function fromString(string $type): self
{
return new self($type);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
use function sprintf;
use function var_export;

final class UnsupportedValue extends InvalidArgumentException implements ClickHouseClientException
final class UnsupportedParamValue extends InvalidArgumentException implements ClickHouseClientException
{
public static function type(mixed $value): self
{
Expand Down
Loading