Conversation
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
# Conflicts: # vortex-array/public-api.lock # vortex-array/src/memory.rs # vortex-io/src/object_store/read_at.rs # vortex-io/src/std_file/read_at.rs
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Merging this PR will degrade performance by 43.39%
Performance Changes
Comparing Footnotes
|
Polar Signals Profiling ResultsLatest Run
Previous Runs (3)
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals ProfilingVortex (geomean): 1.044x ➖ datafusion / vortex-file-compressed (1.044x ➖, 0↑ 2↓)
|
File Sizes: PolarSignals ProfilingNo file size changes detected. |
Benchmarks: TPC-H SF=1 on NVMEVerdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.832x ✅, 21↑ 0↓)
datafusion / vortex-compact (0.852x ✅, 19↑ 0↓)
datafusion / parquet (0.839x ✅, 18↑ 0↓)
datafusion / arrow (0.753x ✅, 21↑ 0↓)
duckdb / vortex-file-compressed (0.836x ✅, 21↑ 0↓)
duckdb / vortex-compact (0.842x ✅, 20↑ 0↓)
duckdb / parquet (0.906x ➖, 13↑ 1↓)
duckdb / duckdb (0.883x ✅, 13↑ 0↓)
Full attributed analysis
|
File Sizes: TPC-H SF=1 on NVMEFile Size Changes (195 files changed, -98.4% overall, 0↑ 195↓)
Totals:
|
Benchmarks: FineWeb NVMeVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (1.052x ➖, 1↑ 2↓)
datafusion / vortex-compact (0.996x ➖, 1↑ 0↓)
datafusion / parquet (1.016x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.061x ➖, 0↑ 2↓)
duckdb / vortex-compact (1.073x ➖, 0↑ 2↓)
duckdb / parquet (1.029x ➖, 0↑ 1↓)
Full attributed analysis
|
File Sizes: FineWeb NVMeNo file size changes detected. |
Benchmarks: TPC-DS SF=1 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (1.054x ➖, 0↑ 6↓)
datafusion / vortex-compact (1.057x ➖, 0↑ 6↓)
datafusion / parquet (1.041x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed (1.043x ➖, 0↑ 12↓)
duckdb / vortex-compact (1.029x ➖, 2↑ 4↓)
duckdb / parquet (1.027x ➖, 1↑ 3↓)
duckdb / duckdb (1.027x ➖, 0↑ 5↓)
Full attributed analysis
|
File Sizes: TPC-DS SF=1 on NVMENo file size changes detected. |
Benchmarks: TPC-H SF=10 on NVMEVerdict: No clear signal (medium confidence) datafusion / vortex-file-compressed (1.441x ❌, 0↑ 22↓)
datafusion / vortex-compact (1.177x ❌, 0↑ 17↓)
datafusion / parquet (1.363x ❌, 0↑ 22↓)
datafusion / arrow (1.160x ❌, 0↑ 11↓)
duckdb / vortex-file-compressed (1.057x ➖, 0↑ 4↓)
duckdb / vortex-compact (1.055x ➖, 0↑ 3↓)
duckdb / parquet (1.007x ➖, 0↑ 0↓)
duckdb / duckdb (1.006x ➖, 0↑ 0↓)
Full attributed analysis
|
File Sizes: TPC-H SF=10 on NVMENo file size changes detected. |
Benchmarks: TPC-H SF=1 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (1.179x ➖, 0↑ 7↓)
datafusion / vortex-compact (1.166x ➖, 0↑ 4↓)
datafusion / parquet (1.217x ➖, 0↑ 5↓)
duckdb / vortex-file-compressed (1.041x ➖, 0↑ 1↓)
duckdb / vortex-compact (1.035x ➖, 0↑ 0↓)
duckdb / parquet (1.044x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: FineWeb S3Verdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.999x ➖, 1↑ 0↓)
datafusion / vortex-compact (1.210x ➖, 0↑ 1↓)
datafusion / parquet (1.146x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.240x ➖, 0↑ 4↓)
duckdb / vortex-compact (1.068x ➖, 0↑ 0↓)
duckdb / parquet (1.051x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: Random AccessVortex (geomean): 0.914x ➖ unknown / unknown (1.001x ➖, 7↑ 6↓)
|
Benchmarks: Statistical and Population GeneticsVerdict: No clear signal (low confidence) duckdb / vortex-file-compressed (0.948x ➖, 1↑ 0↓)
duckdb / vortex-compact (1.004x ➖, 0↑ 0↓)
duckdb / parquet (0.992x ➖, 0↑ 0↓)
Full attributed analysis
|
File Sizes: Statistical and Population GeneticsNo file size changes detected. |
Benchmarks: TPC-H SF=10 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (1.086x ➖, 0↑ 3↓)
datafusion / vortex-compact (1.172x ➖, 0↑ 6↓)
datafusion / parquet (1.182x ➖, 0↑ 4↓)
duckdb / vortex-file-compressed (1.056x ➖, 0↑ 1↓)
duckdb / vortex-compact (1.070x ➖, 0↑ 2↓)
duckdb / parquet (1.098x ➖, 0↑ 1↓)
Full attributed analysis
|
Benchmarks: Clickbench on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (1.037x ➖, 1↑ 3↓)
datafusion / parquet (1.014x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed (0.963x ➖, 6↑ 0↓)
duckdb / parquet (1.006x ➖, 0↑ 0↓)
duckdb / duckdb (0.998x ➖, 2↑ 0↓)
Full attributed analysis
|
File Sizes: Clickbench on NVMEFile Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Benchmarks: CompressionVortex (geomean): 1.005x ➖ unknown / unknown (0.996x ➖, 0↑ 0↓)
|
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Adds a relatively simple thread-local implementation of a host allocator as the default vortex allocator.