This page explains how to get RGB to XYZ conversion matrix. It's written in Japanese but you can see equations.
The page is not copiable so I'll just write several comments. It may help you understand it.
You need x and y value of r, g, b, and w(white point).
(xr, yr, zr), (xg, yg, zg), (xb, yb, zb), (xw, yw, zw) are the (x, y, z) value of r, g, b, and w.
z value can be obtained from x and y (See below).
Sr, Sg, Sb are the variables. You can get the conversion matrix by calculating these values.
X, Y, Z indicates a color including intensity
x, y, z indicates a color excluding intensity, which are
x = X / (X + Y + Z)
y = Y / (X + Y + Z)
z = Z / (X + Y + Z)
x + y + z = 1
Same color with different intensity produces the same (x, y, z) value.
So if you know x, y, and Y,
X = x * (X + Y + Z) = x * Y/y
Like the second equation, if you multiply M and
R = |0|
|X| |xrSr xgSg xbSb| |1| |xrSr| |xr|
|Y| = MR = |yrSr ygSg ybSb| |0| = |yrSr| = Sr |yr|
|Z| |zrSr zgSg zbSb| |0| |zrSr| |zr|
The author of the page recommends you to use ITU-T BT.709 if you don't know the exact value of (xr, yr, zr), (xg, yg), (xb, yb), (xw, yw) are the (x, y).
(xw, yw, xr, yr, xg, yg, xb, yb)=(0.3127, 0.3290, 0.64, 0.33, 0.30, 0.60, 0.15, 0.06)
xW and yW are the ones for D65.
The conversion matrix is shown on the page (eq4).
It is not almighty. If the RGB space is not compatible with ITU-T BT.709, the result will include error.
The white point (xw, yw) for a color templature can be calculated with Plank's law (I don't want to calculate it, anybody knows the answer table?).
By converting RGB to XYZ with a (xw, yw) and converting the XYZ to RGB with another (xw, yw), you can convert RGB value for one color templature to another.