KDB knowledge

Correlation and regression

Do two series move together? corr, covariance, a straight trend line (regr_slope, regr_intercept, regr_r2), and the beta and alpha of a share against the market.

These aggregates look at pairs of numbers: one y and one x on each row, for example the return of CITY and the return of a market index on the same day. They answer two questions:

They follow these rules:

Use returns, not prices. Two share prices that both trend up for months look strongly correlated even when their daily moves have nothing to do with each other. Compare the daily changes (pct_change(close), or pct_return). To put two series side by side, make each one in a subquery and join them on time.

Example: Which shares move together? Correlation of the daily % changes of every pair (sample data: sample)

SELECT a.symbol AS share_a, b.symbol AS share_b, count(*) AS days,
       round(corr(a.r, b.r), 3) AS correlation
FROM (SELECT time, symbol, pct_change(close) OVER (PARTITION BY symbol ORDER BY time) AS r
      FROM ochl_day WHERE time >= '2026-08-23') a
JOIN (SELECT time, symbol, pct_change(close) OVER (PARTITION BY symbol ORDER BY time) AS r
      FROM ochl_day WHERE time >= '2026-08-23') b
  ON a.time = b.time AND a.symbol < b.symbol
GROUP BY a.symbol, b.symbol
ORDER BY correlation DESC;

Result:

share_a (SYMBOL)share_b (SYMBOL)days (LONG)correlation (DOUBLE)
COASTVOLT150.366
CITYVOLT150.217
CITYCOAST150.032

(3 rows)

corr

Syntax:

corr(y, x)
corr(y, x) OVER (...)

The Pearson correlation of y and x: a number from -1 to 1 that tells how well the points lie on one straight line. Above about 0.7 is a strong link, below about 0.3 a weak one.

ArgumentTypeMeaning
ynumberthe first value of each pair
xnumberthe second value of each pair

Example: CITY and COAST: daily returns joined on the day (sample data: sample)

SELECT count(*) AS days, regr_count(a.r, b.r) AS pairs, round(corr(a.r, b.r), 3) AS correlation
FROM (SELECT time, pct_change(close) AS r FROM ochl_day WHERE symbol = 'CITY' AND time >= '2026-08-23') a
JOIN (SELECT time, pct_change(close) AS r FROM ochl_day WHERE symbol = 'COAST' AND time >= '2026-08-23') b
  ON a.time = b.time;

Result:

days (LONG)pairs (LONG)correlation (DOUBLE)
15140.032

(1 row)

Example: Moving 5-day correlation (with OVER) (sample data: sample)

SELECT time, round(corr(city_r, coast_r) OVER (ORDER BY time ROWS 4 PRECEDING), 3) AS corr_5d
FROM (SELECT a.time AS time, a.r AS city_r, b.r AS coast_r
      FROM (SELECT time, pct_change(close) AS r FROM ochl_day WHERE symbol = 'CITY' AND time >= '2026-08-23') a
      JOIN (SELECT time, pct_change(close) AS r FROM ochl_day WHERE symbol = 'COAST' AND time >= '2026-08-23') b
        ON a.time = b.time) t
ORDER BY time;

Result:

time (TIMESTAMP)corr_5d (DOUBLE)
2026-08-23T00:00:00.000ZNULL
2026-08-24T00:00:00.000ZNULL
2026-08-25T00:00:00.000Z1
2026-08-26T00:00:00.000Z0.939
2026-08-27T00:00:00.000Z0.326
2026-08-30T00:00:00.000Z-0.208
2026-08-31T00:00:00.000Z-0.149
2026-09-01T00:00:00.000Z-0.091
2026-09-02T00:00:00.000Z-0.063
2026-09-03T00:00:00.000Z-0.331
2026-09-06T00:00:00.000Z-0.174
2026-09-07T00:00:00.000Z0.584
2026-09-08T00:00:00.000Z0.293
2026-09-09T00:00:00.000Z0.386
2026-09-10T00:00:00.000Z0.424

(15 rows)

Example: Why returns: today's price and yesterday's price look strongly linked (sample data: sample)

SELECT round(corr(close, prev_close), 3) AS price_corr
FROM (SELECT close, lag(close) AS prev_close
      FROM ochl_day
      WHERE symbol = 'CITY' AND time >= '2026-08-23') t;

Result:

price_corr (DOUBLE)
0.78

(1 row)

covar_pop

Syntax:

covar_pop(y, x)

The population covariance: the average of (y - average y) × (x - average x). It is positive when y and x are often above their averages at the same time, negative when one is above while the other is below. It divides by n, the number of pairs.

ArgumentTypeMeaning
ynumberthe first value of each pair
xnumberthe second value of each pair

Example: Pairs with a NULL are skipped: only 3 pairs count (sample data: sample)

SELECT regr_count(y, x) AS pairs, covar_pop(y, x) AS covar_pop, covar_samp(y, x) AS covar_samp
FROM (SELECT 1 AS x, 2 AS y UNION ALL SELECT 2, 4 UNION ALL SELECT 3, 5
      UNION ALL SELECT 4, NULL UNION ALL SELECT NULL, 9) t;

Result:

pairs (LONG)covar_pop (DOUBLE)covar_samp (DOUBLE)
311.5

(1 row)

covar_samp

Syntax:

covar_samp(y, x)

The sample covariance: like covar_pop, but it divides by n - 1. Use it when your rows are only a part of a longer history, as for returns (see "sample or population" in statistics).

ArgumentTypeMeaning
ynumberthe first value of each pair
xnumberthe second value of each pair

Example: Covariance of price and volume, minute by minute (sample data: sample)

SELECT round(covar_samp(close, volume), 1) AS covar_samp,
       round(covar_pop(close, volume), 1) AS covar_pop,
       round(corr(close, volume), 3) AS corr
FROM ochl_minutes
WHERE symbol = 'CITY' AND time IN '2026-09-10';

Result:

covar_samp (DOUBLE)covar_pop (DOUBLE)corr (DOUBLE)
0.80.80.001

(1 row)

Example: One pair: covar_pop is 0, covar_samp and corr are NULL (sample data: sample)

SELECT covar_pop(y, x) AS covar_pop, covar_samp(y, x) AS covar_samp, corr(y, x) AS corr
FROM (SELECT 1 AS x, 5 AS y) t;

Result:

covar_pop (DOUBLE)covar_samp (STRING)corr (STRING)
0NULLNULL

(1 row)

regr_slope

Syntax:

regr_slope(y, x)

The slope of the best straight line through the points: how much y goes up when x goes up by 1. With x = the day number and y = the close, it is the trend in dollars per day.

ArgumentTypeMeaning
ynumberthe value to explain, for example close
xnumberthe value that explains it, for example the day or minute number

Example: Trend line of VOLT since 23 August: dollars per trading day (sample data: sample)

SELECT round(regr_slope(close, day_no), 3) AS dollars_per_day,
       round(regr_intercept(close, day_no), 2) AS line_at_day_0,
       round(regr_r2(close, day_no), 3) AS r2
FROM (SELECT row_number() AS day_no, close
      FROM ochl_day
      WHERE symbol = 'VOLT' AND time >= '2026-08-23') t;

Result:

dollars_per_day (DOUBLE)line_at_day_0 (DOUBLE)r2 (DOUBLE)
0.673172.210.247

(1 row)

Example: Intraday trend of each share on 10 September: dollars per minute (sample data: sample)

SELECT symbol, round(regr_slope(close, minute_no), 4) AS dollars_per_minute, round(regr_r2(close, minute_no), 3) AS r2
FROM (SELECT symbol, close, row_number() OVER (PARTITION BY symbol ORDER BY time) AS minute_no
      FROM ochl_minutes
      WHERE time IN '2026-09-10') t
GROUP BY symbol
ORDER BY symbol;

Result:

symbol (SYMBOL)dollars_per_minute (DOUBLE)r2 (DOUBLE)
CITY-0.00310.021
COAST-0.0280.848

(2 rows)

regr_intercept

Syntax:

regr_intercept(y, x)

The intercept of the best straight line: the value of the line where x is 0. Together with the slope it gives the line: y = regr_slope × x + regr_intercept.

ArgumentTypeMeaning
ynumberthe value to explain
xnumberthe value that explains it

Example: Where would the VOLT trend line be on day 16? (sample data: sample)

SELECT round(regr_intercept(close, day_no) + regr_slope(close, day_no) * 16, 2) AS line_day_16,
       max(day_no) AS last_day
FROM (SELECT row_number() AS day_no, close
      FROM ochl_day
      WHERE symbol = 'VOLT' AND time >= '2026-08-23') t;

Result:

line_day_16 (DOUBLE)last_day (LONG)
182.9715

(1 row)

regr_r2

Syntax:

regr_r2(y, x)

The R squared (coefficient of determination), from 0 to 1: how much of the ups and downs of y the straight line explains. 1: all points are on the line. 0: the line explains nothing. For a trend line, a high R squared means a clean, steady trend.

ArgumentTypeMeaning
ynumberthe value to explain
xnumberthe value that explains it

Example: A flat y: R squared 1, slope 0, corr NULL (sample data: sample)

SELECT regr_r2(y, x) AS r2, regr_slope(y, x) AS slope, corr(y, x) AS corr
FROM (SELECT 1 AS x, 5 AS y UNION ALL SELECT 2, 5) t;

Result:

r2 (DOUBLE)slope (DOUBLE)corr (STRING)
10NULL

(1 row)

regr_count

Syntax:

regr_count(y, x)

The number of pairs where both y and x are not NULL: the rows the other regr_ functions really use.

ArgumentTypeMeaning
yany numberthe first value of each pair
xany numberthe second value of each pair

Example: The first day of a return series has no return: 15 days, 14 pairs (sample data: sample)

SELECT count(*) AS days, regr_count(a.r, b.r) AS pairs
FROM (SELECT time, pct_change(close) AS r FROM ochl_day WHERE symbol = 'CITY' AND time >= '2026-08-23') a
JOIN (SELECT time, pct_change(close) AS r FROM ochl_day WHERE symbol = 'VOLT' AND time >= '2026-08-23') b
  ON a.time = b.time;

Result:

days (LONG)pairs (LONG)
1514

(1 row)

Example: No rows: regr_count is 0, corr is NULL (sample data: sample)

SELECT regr_count(close, volume) AS pairs, corr(close, volume) AS corr
FROM ochl_day
WHERE symbol = 'NONE';

Result:

pairs (LONG)corr (STRING)
0NULL

(1 row)

regr_avgx, regr_avgy

Syntax:

regr_avgx(y, x)
regr_avgy(y, x)

The average of x (regr_avgx) or of y (regr_avgy), taken only over the pairs where both values are not NULL. So it can be different from avg(x), which also counts rows where y is NULL.

ArgumentTypeMeaning
ynumberthe first value of each pair
xnumberthe second value of each pair

Example: avg counts every value, regr_avgx only the full pairs (sample data: sample)

SELECT avg(x) AS avg_x, regr_avgx(y, x) AS regr_avgx, avg(y) AS avg_y, round(regr_avgy(y, x), 4) AS regr_avgy
FROM (SELECT 1 AS x, 2 AS y UNION ALL SELECT 2, 4 UNION ALL SELECT 3, 5
      UNION ALL SELECT 4, NULL UNION ALL SELECT NULL, 9) t;

Result:

avg_x (DOUBLE)regr_avgx (DOUBLE)avg_y (DOUBLE)regr_avgy (DOUBLE)
2.5253.6667

(1 row)

regr_sxx, regr_syy, regr_sxy

Syntax:

regr_sxx(y, x)
regr_syy(y, x)
regr_sxy(y, x)

The sums that the regression is made from, over the full pairs:

FunctionSum
regr_sxx(y, x)sum((x - avg x)²)
regr_syy(y, x)sum((y - avg y)²)
regr_sxy(y, x)sum((x - avg x) × (y - avg y))
ArgumentTypeMeaning
ynumberthe first value of each pair
xnumberthe second value of each pair

Example: Build the slope from the sums (sample data: sample)

SELECT regr_sxx(y, x) AS sxx, round(regr_syy(y, x), 4) AS syy, regr_sxy(y, x) AS sxy,
       regr_sxy(y, x) / regr_sxx(y, x) AS slope_by_hand, regr_slope(y, x) AS slope
FROM (SELECT 1 AS x, 2 AS y UNION ALL SELECT 2, 4 UNION ALL SELECT 3, 5) t;

Result:

sxx (DOUBLE)syy (DOUBLE)sxy (DOUBLE)slope_by_hand (DOUBLE)slope (DOUBLE)
24.666731.51.5

(1 row)

beta

Syntax:

beta(stock_return, market_return)
beta(stock_return, market_return) OVER (...)

The beta of a share: how strongly it follows the market. It is the regression slope of the share's returns (y) against the market's returns (x), for example a stock market index.

ArgumentTypeMeaning
stock_returnnumberthe return of the share on each day (y)
market_returnnumberthe return of the market on the same day (x)

Example: Beta of each share against the market index (sample data: sample, analysis-market)

SELECT s.symbol, regr_count(s.r, m.r) AS days,
       round(beta(s.r, m.r), 3) AS beta,
       round(corr(s.r, m.r), 3) AS corr,
       round(regr_r2(s.r, m.r), 3) AS r2
FROM (SELECT time, symbol, pct_change(close) OVER (PARTITION BY symbol ORDER BY time) AS r
      FROM ochl_day WHERE time >= '2026-08-23') s
JOIN (SELECT time, pct_change(close) AS r
      FROM market_index WHERE time >= '2026-08-23') m
  ON s.time = m.time
GROUP BY s.symbol
ORDER BY beta DESC;

Result:

symbol (SYMBOL)days (LONG)beta (DOUBLE)corr (DOUBLE)r2 (DOUBLE)
VOLT143.3220.7050.497
COAST141.9620.6860.47
CITY140.8620.4620.214

(3 rows)

alpha

Syntax:

alpha(stock_return, market_return)
alpha(stock_return, market_return, risk_free)

The alpha (Jensen's alpha) of a share: the part of its average return that the market does not explain. After taking away what its beta would give, a positive alpha means the share did better than the market risk predicts; a negative alpha means worse. Fund managers try to "make alpha".

It is (avg(stock) - risk_free) - beta × (avg(market) - risk_free).

ArgumentTypeMeaning
stock_returnnumberthe return of the share on each day (y)
market_returnnumberthe return of the market on the same day (x)
risk_freenumberthe return without risk for one period (for example a fixed deposit rate per day), in the same unit as the returns; default 0

Example: Alpha and beta of each share (daily returns in %) (sample data: sample, analysis-market)

SELECT s.symbol,
       round(beta(s.r, m.r), 3) AS beta,
       round(alpha(s.r, m.r), 4) AS alpha_pct_per_day,
       round(avg(s.r), 4) AS avg_return_pct
FROM (SELECT time, symbol, pct_change(close) OVER (PARTITION BY symbol ORDER BY time) AS r
      FROM ochl_day WHERE time >= '2026-08-23') s
JOIN (SELECT time, pct_change(close) AS r
      FROM market_index WHERE time >= '2026-08-23') m
  ON s.time = m.time
GROUP BY s.symbol
ORDER BY alpha_pct_per_day DESC;

Result:

symbol (SYMBOL)beta (DOUBLE)alpha_pct_per_day (DOUBLE)avg_return_pct (DOUBLE)
VOLT3.3220.41360.4088
CITY0.862-0.0385-0.0397
COAST1.962-0.2682-0.271

(3 rows)

Example: With a risk-free return of 0.024% a day (sample data: sample)

SELECT round(alpha(y, x), 5) AS alpha, round(alpha(y, x, 0.024), 5) AS alpha_with_risk_free, round(beta(y, x), 3) AS beta
FROM (SELECT 1.0 AS x, 2.0 AS y UNION ALL SELECT 2.0, 3.0 UNION ALL SELECT -1.0, -1.0) t;

Result:

alpha (DOUBLE)alpha_with_risk_free (DOUBLE)beta (DOUBLE)
0.428570.437141.357

(1 row)

Example: Returns must be numbers (sample data: sample; shows an error)

SELECT beta(symbol, close) FROM ochl_day;

Result:

Error: line 1, column 8: beta: needs numbers, got 'CITY'