In [27]:
plt.plot(real_eth_price,color = 'red', label = 'Real Price')
plt.plot(pred, color = 'blue', label = 'Predicted Price')
plt.title('ETH Price Prediction')
plt.xlabel('Time')
plt.ylabel('ETH Price')
plt.legend()
plt.show()
This article explores the long short-term memory (LSTM) network and how it can be used to make ETH market predictions comparing to ARMA model.
Data set introduction: The Ethereum price dataset is collected from https://www.investing.com/crypto/ethereum/historical-data. The dates are from Jan 2016 to October 2021. The dataset is split into trainset and testset with a 0.85: 0.15 proportion
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
dataset_train = pd.read_csv("./ETH_trainset.csv")
dataset_train
| Date | Open | High | Low | Close | Volume | Transaction | Avg | Turnover | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 2016/1/4 | 2.201887 | 2.203713 | 2.074030 | 2.080422 | 298546954 | 703760451 | 2.152831 | 2.406860 |
| 1 | 2016/1/5 | 2.063984 | 2.110560 | 2.036586 | 2.087729 | 645699581 | 1473642292 | 2.084296 | 5.069479 |
| 2 | 2016/1/6 | 2.091382 | 2.118780 | 2.077683 | 2.116040 | 400953404 | 921398473 | 2.098705 | 3.178638 |
| 3 | 2016/1/7 | 2.087729 | 2.087729 | 1.977223 | 1.988182 | 89166922 | 199851686 | 2.046925 | 0.706441 |
| 4 | 2016/1/8 | 2.042065 | 2.063070 | 1.969004 | 2.026540 | 576676937 | 1279000234 | 2.025517 | 4.563040 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 1195 | 2020/12/1 | 3.476000 | 3.563000 | 3.476000 | 3.557000 | 811632065 | 2864395143 | 3.529179 | 5.120524 |
| 1196 | 2020/12/2 | 3.558000 | 3.573000 | 3.525000 | 3.545000 | 541549798 | 1922213431 | 3.549468 | 3.377095 |
| 1197 | 2020/12/3 | 3.544000 | 3.552000 | 3.521000 | 3.538000 | 545270803 | 1927642645 | 3.535202 | 3.333141 |
| 1198 | 2020/12/4 | 3.535000 | 3.549000 | 3.501000 | 3.543000 | 442716436 | 1560214891 | 3.524186 | 2.713261 |
| 1199 | 2020/12/7 | 3.544000 | 3.549000 | 3.492000 | 3.511000 | 486309440 | 1708975464 | 3.514173 | 3.011658 |
1200 rows × 9 columns
Statistic description
dataset_train.describe()
| Open | High | Low | Close | Volume | Transaction | Avg | Turnover | |
|---|---|---|---|---|---|---|---|---|
| count | 1200.000000 | 1200.000000 | 1200.000000 | 1200.000000 | 1.200000e+03 | 1.200000e+03 | 1200.000000 | 1200.000000 |
| mean | 2.541748 | 2.561777 | 2.524633 | 2.543779 | 4.880979e+08 | 1.336211e+09 | 2.543682 | 3.364845 |
| std | 0.402380 | 0.405934 | 0.399084 | 0.402597 | 3.267784e+08 | 9.434541e+08 | 0.402112 | 1.960748 |
| min | 1.751646 | 1.785437 | 1.726075 | 1.748906 | 7.785455e+07 | 1.719694e+08 | 1.754479 | 0.636029 |
| 25% | 2.208369 | 2.218636 | 2.199316 | 2.207904 | 2.591224e+08 | 6.586369e+08 | 2.209953 | 1.976679 |
| 50% | 2.543266 | 2.559785 | 2.531317 | 2.545158 | 4.163454e+08 | 1.162825e+09 | 2.545532 | 2.988435 |
| 75% | 2.832092 | 2.856339 | 2.816171 | 2.835189 | 6.238289e+08 | 1.762519e+09 | 2.833810 | 4.276485 |
| max | 3.558000 | 3.575000 | 3.525000 | 3.557000 | 2.599900e+09 | 7.843001e+09 | 3.549468 | 14.991264 |
Using open price as the predicting target
trainset = dataset_train.iloc[:,1:2].values
Min_max scaler is applied for faster trainning speed.
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler(feature_range = (0,1))
training_scaled = scaler.fit_transform(trainset)
x_train = []
y_train = []
Since this is a time series data, I assume the datapoint can be predicted by previous 60 datapoints. Why 60? In stock market, 60 day MACD line is an important feature to predict the price of a stock. I tried 20 datapoints as features (MACD 20 line is also important in stock analysis.), but the results are bad comparing to 60 datapoints.
for i in range(60,trainset.shape[0]):
x_train.append(training_scaled[i-60:i, 0])
y_train.append(training_scaled[i,0])
x_train,y_train = np.array(x_train),np.array(y_train)
x_train = np.reshape(x_train, (x_train.shape[0],x_train.shape[1],1))
LSTM (long short-term memory network) is a variant of RNN. RNN and conventional neural network is different from the layer of neurons between the establishment of a connection. The advantage is that it is convenient to use the sequence of before and after correlation analysis of time series. However, RNN ,due to gradient disappearance reasons, can only have short-term memory. LSTM network uses gate control of short-term memory and long-term memory combined, to solve the problem of gradient disappearance, resulting in better prediction sequence situation.
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import Dropout
model = Sequential()
model.add(LSTM(units = 60,return_sequences = True,input_shape = (x_train.shape[1],1)))
model.add(Dropout(0.4))
model.add(LSTM(units = 60,return_sequences = True))
model.add(Dropout(0.4))
model.add(LSTM(units = 60,return_sequences = True))
model.add(Dropout(0.4))
model.add(LSTM(units = 60,return_sequences = True))
model.add(Dropout(0.4))
model.add(LSTM(units = 60))
model.add(Dropout(0.4))
model.add(Dense(units = 1))
model.compile(optimizer = 'adam',loss = 'mean_squared_error')
model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= lstm (LSTM) (None, 60, 60) 14880 _________________________________________________________________ dropout (Dropout) (None, 60, 60) 0 _________________________________________________________________ lstm_1 (LSTM) (None, 60, 60) 29040 _________________________________________________________________ dropout_1 (Dropout) (None, 60, 60) 0 _________________________________________________________________ lstm_2 (LSTM) (None, 60, 60) 29040 _________________________________________________________________ dropout_2 (Dropout) (None, 60, 60) 0 _________________________________________________________________ lstm_3 (LSTM) (None, 60, 60) 29040 _________________________________________________________________ dropout_3 (Dropout) (None, 60, 60) 0 _________________________________________________________________ lstm_4 (LSTM) (None, 60) 29040 _________________________________________________________________ dropout_4 (Dropout) (None, 60) 0 _________________________________________________________________ dense (Dense) (None, 1) 61 ================================================================= Total params: 131,101 Trainable params: 131,101 Non-trainable params: 0 _________________________________________________________________
model.fit(x_train,y_train,epochs = 100, batch_size = 32)
Epoch 1/100 36/36 [==============================] - 3s 81ms/step - loss: 0.0319 Epoch 2/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0100 Epoch 3/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0088 Epoch 4/100 36/36 [==============================] - 3s 78ms/step - loss: 0.0083 Epoch 5/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0080 Epoch 6/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0074 Epoch 7/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0077 Epoch 8/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0073 Epoch 9/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0079 Epoch 10/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0066 Epoch 11/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0076 Epoch 12/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0063 Epoch 13/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0061 Epoch 14/100 36/36 [==============================] - 3s 83ms/step - loss: 0.0059 Epoch 15/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0063 Epoch 16/100 36/36 [==============================] - 3s 82ms/step - loss: 0.0055 Epoch 17/100 36/36 [==============================] - 3s 83ms/step - loss: 0.0061 Epoch 18/100 36/36 [==============================] - 3s 82ms/step - loss: 0.0056 Epoch 19/100 36/36 [==============================] - 3s 83ms/step - loss: 0.0055 Epoch 20/100 36/36 [==============================] - 3s 82ms/step - loss: 0.0046 Epoch 21/100 36/36 [==============================] - 3s 81ms/step - loss: 0.0047 Epoch 22/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0046 Epoch 23/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0049 Epoch 24/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0049 Epoch 25/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0045 Epoch 26/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0045 Epoch 27/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0048 Epoch 28/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0039 Epoch 29/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0045 Epoch 30/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0049 Epoch 31/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0040 Epoch 32/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0041 Epoch 33/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0040 Epoch 34/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0035 Epoch 35/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0037 Epoch 36/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0035 Epoch 37/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0033 Epoch 38/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0036 Epoch 39/100 36/36 [==============================] - 3s 78ms/step - loss: 0.0036 Epoch 40/100 36/36 [==============================] - 3s 78ms/step - loss: 0.0030 Epoch 41/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0034 Epoch 42/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0033 Epoch 43/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0030 Epoch 44/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0033 Epoch 45/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0029 Epoch 46/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0032 Epoch 47/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0029 Epoch 48/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0031 Epoch 49/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0029 Epoch 50/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0029 Epoch 51/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0027 Epoch 52/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0029 Epoch 53/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0029 Epoch 54/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0027 Epoch 55/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0029 Epoch 56/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0027 Epoch 57/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0027 Epoch 58/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0026 Epoch 59/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0027 Epoch 60/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0026 Epoch 61/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0024 Epoch 62/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0026 Epoch 63/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0028 Epoch 64/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0025 Epoch 65/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0026 Epoch 66/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0025 Epoch 67/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0026 Epoch 68/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0025 Epoch 69/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0023 Epoch 70/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0024 Epoch 71/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0024 Epoch 72/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0021 Epoch 73/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0022 Epoch 74/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0023 Epoch 75/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0026 Epoch 76/100 36/36 [==============================] - 3s 81ms/step - loss: 0.0022 Epoch 77/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0023 Epoch 78/100 36/36 [==============================] - 3s 81ms/step - loss: 0.0020 Epoch 79/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0022 Epoch 80/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0021 Epoch 81/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0022 Epoch 82/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0020 Epoch 83/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0019 Epoch 84/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0021 Epoch 85/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0021 Epoch 86/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0024 Epoch 87/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0019 Epoch 88/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0020 Epoch 89/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0020 Epoch 90/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0020 Epoch 91/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0018 Epoch 92/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0018 Epoch 93/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0023 Epoch 94/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0019 Epoch 95/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0020 Epoch 96/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0019 Epoch 97/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0017 Epoch 98/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0020 Epoch 99/100 36/36 [==============================] - 3s 80ms/step - loss: 0.0019 Epoch 100/100 36/36 [==============================] - 3s 79ms/step - loss: 0.0017
<tensorflow.python.keras.callbacks.History at 0x1a99f3b5518>
dataset_test = pd.read_csv("./ETH_testset.csv")
dataset_test
| Date | Open | High | Low | Close | Volume | Transaction | Avg | Turnover | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 2020/12/8 | 3.580 | 3.580 | 3.488 | 3.499 | 436983773 | 1535311280 | 3.513429 | 2.675029 |
| 1 | 2020/12/9 | 3.507 | 3.520 | 3.465 | 3.470 | 512286103 | 1791501940 | 3.497073 | 3.118815 |
| 2 | 2020/12/10 | 3.461 | 3.477 | 3.439 | 3.456 | 458333410 | 1586191032 | 3.460780 | 2.805722 |
| 3 | 2020/12/11 | 3.469 | 3.478 | 3.403 | 3.429 | 540670655 | 1857613082 | 3.435757 | 3.336780 |
| 4 | 2020/12/14 | 3.435 | 3.466 | 3.430 | 3.461 | 303408537 | 1046907314 | 3.450487 | 1.874690 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 198 | 2021/9/29 | 3.225 | 3.260 | 3.195 | 3.248 | 767053532 | 2479461267 | 3.232449 | 4.634802 |
| 199 | 2021/9/30 | 3.248 | 3.250 | 3.222 | 3.231 | 521097066 | 1685853875 | 3.235201 | 3.188529 |
| 200 | 2021/10/8 | 3.262 | 3.300 | 3.258 | 3.295 | 734601345 | 2416701710 | 3.289814 | 4.527850 |
| 201 | 2021/10/11 | 3.309 | 3.353 | 3.306 | 3.312 | 881909167 | 2932563131 | 3.325244 | 5.568007 |
| 202 | 2021/10/12 | 3.300 | 3.322 | 3.271 | 3.295 | 703599890 | 2318202782 | 3.294774 | 4.442236 |
203 rows × 9 columns
dataset_total = pd.concat((dataset_train['Open'],dataset_test['Open']),axis = 0)
real_eth_price = dataset_test.iloc[:,1:2].values
input_data = dataset_total[len(dataset_total) - len(dataset_test)-60:].values
input_data.shape
(263,)
input_data = input_data.reshape(-1,1)
input_data = scaler.transform(input_data)
input_data.shape
(263, 1)
x_test = []
for i in range(60,258):
x_test.append(input_data[i-60:i,0])
x_test = np.array(x_test)
x_test = np.reshape(x_test, (x_test.shape[0],x_test.shape[1],1))
pred = model.predict(x_test)
pred = scaler.inverse_transform(pred)
plt.plot(real_eth_price,color = 'red', label = 'Real Price')
plt.plot(pred, color = 'blue', label = 'Predicted Price')
plt.title('ETH Price Prediction')
plt.xlabel('Time')
plt.ylabel('ETH Price')
plt.legend()
plt.show()
Comparing to ARMA model
# reference: https://towardsdatascience.com/time-series-forecasting-predicting-stock-prices-using-an-arima-model-2e3b3080bd70
from statsmodels.tsa.arima_model import ARIMA
from sklearn.metrics import mean_squared_error
dataset_total = pd.concat((dataset_train['Open'],dataset_test['Open']),axis = 0)
print(dataset_total.shape)
input_data = dataset_total[:].values
input_data = input_data.reshape(-1,1)
train_data, test_data = input_data[0:int(len(input_data)*0.85)], input_data[int(len(input_data)*0.85):]
training_data = train_data
test_data = test_data
history = [x for x in training_data]
model_predictions = []
N_test_observations = len(test_data)
for time_point in range(N_test_observations):
model = ARIMA(history, order=(4,1,0))
model_fit = model.fit(disp=0)
output = model_fit.forecast()
yhat = output[0]
model_predictions.append(yhat)
true_test_value = test_data[time_point]
history.append(true_test_value)
MSE_error = mean_squared_error(test_data, model_predictions)
print('Testing Mean Squared Error is {}'.format(MSE_error))
plt.plot(test_data,color = 'red', label = 'Real Price')
plt.plot(model_predictions, color = 'blue', label = 'Predicted Price')
plt.title('ETH Price Prediction')
plt.xlabel('Time')
plt.ylabel('ETH Price')
plt.legend()
plt.show()
(1403,)
C:\Users\12820\anaconda3\envs\tf\lib\site-packages\statsmodels\tsa\arima_model.py:472: FutureWarning:
statsmodels.tsa.arima_model.ARMA and statsmodels.tsa.arima_model.ARIMA have
been deprecated in favor of statsmodels.tsa.arima.model.ARIMA (note the .
between arima and model) and
statsmodels.tsa.SARIMAX. These will be removed after the 0.12 release.
statsmodels.tsa.arima.model.ARIMA makes use of the statespace framework and
is both well tested and maintained.
To silence this warning and continue using ARMA and ARIMA until they are
removed, use:
import warnings
warnings.filterwarnings('ignore', 'statsmodels.tsa.arima_model.ARMA',
FutureWarning)
warnings.filterwarnings('ignore', 'statsmodels.tsa.arima_model.ARIMA',
FutureWarning)
warnings.warn(ARIMA_DEPRECATION_WARN, FutureWarning)
Testing Mean Squared Error is 0.002187911583310571
ARMA fitted results are good, following a true distribution (and proved by a very low MSE). In fact, it doesn't make sense to rely on the eth market price of the next day alone. Personally, what I want is not the exact stock price the next day, but whether the stock market price will rise or fall for the next 30 days. But if the task is to predict the next 2 days window, instead of just predicting the next day), ARMA will not be suitable. The long short-term memory model is an extremely powerful time series model. They can predict any future step.