A.I
선형회귀와 로지스틱회귀 본문
회귀(Regression)¶
1. 선형 회귀 분석(Linear Regression)¶
- 종속변수 Y와 한 개 이상의 독립변수 X와의 선형 상관관계를 모델링하는 회귀분석 기법
- 주어진 데이터에 우리의 선형 식이 잘 맞도록 회귀계수 및 오차를 구하는 것
- 머신러닝에서의 선형회귀모델 H = Wx + b
가정(Hypothesis), 가중치(Weight), 편향(bias) - 손실함수 중 최소제곱법이란 n 개의 점 데이터에 대하여 잔차의 제곱의 합을 최소로 하는 W, b를 구하는 방법
- 잔차란 회귀모델을 이용해 추정한 값과 실제 데이터의 차이를 뜻함.
- 회귀모델이 잘 결정되었는지 R2-Score를 통해 알 수 있음(1에 가까울수록 회귀모델이 데이터를 잘 표현하고 있음)
1-1. 최소제곱법¶
In [2]:
# boston house prices dataset에 대한 정보
from sklearn.datasets import load_boston
from sklearn import model_selection
import pandas as pd
import numpy as np
# 데이터 로드
boston = load_boston()
data, price = boston['data'], boston['target']
x_train, x_test, y_train, y_test = model_selection.train_test_split(data, price, test_size=0.2)
df = pd.DataFrame(x_train, columns=boston['feature_names'])
print("boston dataset의 차원: ", data.shape)
print("price의 차원", price.shape)
print("boston train dataset의 차원: ", x_train.shape)
print("boston test dataset의 차원: ", x_test.shape)
print("Boston dataset의 예시")
df.head() # data 요약
boston dataset의 차원: (506, 13) price의 차원 (506,) boston train dataset의 차원: (404, 13) boston test dataset의 차원: (102, 13) Boston dataset의 예시
Out[2]:
CRIM | ZN | INDUS | CHAS | NOX | RM | AGE | DIS | RAD | TAX | PTRATIO | B | LSTAT | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 25.04610 | 0.0 | 18.10 | 0.0 | 0.693 | 5.987 | 100.0 | 1.5888 | 24.0 | 666.0 | 20.2 | 396.90 | 26.77 |
1 | 0.04379 | 80.0 | 3.37 | 0.0 | 0.398 | 5.787 | 31.1 | 6.6115 | 4.0 | 337.0 | 16.1 | 396.90 | 10.24 |
2 | 15.86030 | 0.0 | 18.10 | 0.0 | 0.679 | 5.896 | 95.4 | 1.9096 | 24.0 | 666.0 | 20.2 | 7.68 | 24.39 |
3 | 0.21719 | 0.0 | 10.59 | 1.0 | 0.489 | 5.807 | 53.8 | 3.6526 | 4.0 | 277.0 | 18.6 | 390.94 | 16.03 |
4 | 4.42228 | 0.0 | 18.10 | 0.0 | 0.584 | 6.003 | 94.5 | 2.5403 | 24.0 | 666.0 | 20.2 | 331.29 | 21.32 |
In [19]:
print(boston["DESCR"])
.. _boston_dataset: Boston house prices dataset --------------------------- **Data Set Characteristics:** :Number of Instances: 506 :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target. :Attribute Information (in order): - CRIM per capita crime rate by town - ZN proportion of residential land zoned for lots over 25,000 sq.ft. - INDUS proportion of non-retail business acres per town - CHAS Charles River dummy variable (= 1 if tract bounds river; 0 otherwise) - NOX nitric oxides concentration (parts per 10 million) - RM average number of rooms per dwelling - AGE proportion of owner-occupied units built prior to 1940 - DIS weighted distances to five Boston employment centres - RAD index of accessibility to radial highways - TAX full-value property-tax rate per $10,000 - PTRATIO pupil-teacher ratio by town - B 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town - LSTAT % lower status of the population - MEDV Median value of owner-occupied homes in $1000's :Missing Attribute Values: None :Creator: Harrison, D. and Rubinfeld, D.L. This is a copy of UCI ML housing dataset. https://archive.ics.uci.edu/ml/machine-learning-databases/housing/ This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University. The Boston house-price data of Harrison, D. and Rubinfeld, D.L. 'Hedonic prices and the demand for clean air', J. Environ. Economics & Management, vol.5, 81-102, 1978. Used in Belsley, Kuh & Welsch, 'Regression diagnostics ...', Wiley, 1980. N.B. Various transformations are used in the table on pages 244-261 of the latter. The Boston house-price data has been used in many machine learning papers that address regression problems. .. topic:: References - Belsley, Kuh & Welsch, 'Regression diagnostics: Identifying Influential Data and Sources of Collinearity', Wiley, 1980. 244-261. - Quinlan,R. (1993). Combining Instance-Based and Model-Based Learning. In Proceedings on the Tenth International Conference of Machine Learning, 236-243, University of Massachusetts, Amherst. Morgan Kaufmann.
In [20]:
# Boston dataset의 각 attribute에 선형회귀 적용하는 예제
import pandas as pd
from sklearn import datasets
from sklearn import model_selection
from sklearn.linear_model import LinearRegression
from sklearn import metrics
from sklearn import datasets
import matplotlib.pyplot as plt
fig = plt.figure(figsize=(10,35))
fig.suptitle('Boston dataset - (X:Y = each attr: price) with R2', fontsize=16, y=0.9)
for i in range(data.shape[1]): # boston dataset에서 i번째 attribute(column)을 살펴볼 거에요.
single_attr, attr_name = data[:, i].reshape(-1, 1), boston['feature_names'][i] # i번째 attribute에 대한 data 및 이름
estimator = LinearRegression() # 선형 회귀 모델이에요.
#x에는 single_attr, y에는 price에 해당하는 데이터를 대입해서 최소제곱법을 이용하여 모델 내에서 W, b를 구하는 과정이에요
estimator.fit(single_attr, price)
#위 fit() 과정을 통해 구한 회귀계수를 기반으로 회귀모델에 X값을 대입했을 때의 예측 Y 값이에요.
pred_price = estimator.predict(single_attr)
score = metrics.r2_score(price, pred_price) # 결정계수를 구하는 함수에요.
# 캔버스 생성
ax = fig.add_subplot(7, 2, i+1)
ax.scatter(single_attr, price) # 실제 데이터에 대한 산포도
ax.plot(single_attr, pred_price, color='red') # 선형회귀모델의 추세선
ax.set_title("{} x price, R2 score={:.3f}".format(attr_name ,score)) #subplot의 제목이에요
ax.set_xlabel(attr_name) # x축
ax.set_ylabel('price') # y축
1-2. 경사 하강법(Gradient Descent Algorithm)¶
- 경사 하강의 최소지점이 손실함수의 최소값을 나타낸다고 생각하여 최적의 그래디언트값을 찾아 최적의 W값을 찾는 방법
2. 로지스틱 회귀분석(Logistic Regression)¶
- [0, 1]사이의 값으로 나타내기때문에 1개 이상의 독립변수가 있을 때 이를 이용하여 데이터가 2개의 범주 중 하나에 속하도록 결정하는 이진 분류(binary classification) 많이 사용
- sigmoid function의 순서
- 실제 데이터를 대입하여 Odds 및 회귀계수를 구한다.
- Log-odds를 계산한 후, 이를 sigmoid function의 입력으로 넣어서 특정 범주에 속할 확률값을 계산한다.
- 설정한 threshold에 맞추어 설정값 이상이면 1, 이하면 0으로 이진분류를 수행한다.
In [4]:
# 로지스틱 회귀분석 예제: 유방암 데이터셋
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
# 데이터 로드
cancer=load_breast_cancer()
# y = 0(Malignant - 악성 종양), y=1(Benign - 양성 종양)
cancer_X, cancer_y= cancer.data, cancer['target']
train_X, test_X, train_y, test_y = train_test_split(cancer_X, cancer_y, test_size=0.1, random_state=10) # train 데이터셋과 test 데이터셋으로 나눔
print("전체 검사자 수: {}".format(len(cancer_X)))
print("Attribute 수: {}".format(len(cancer_X[0])))
print("Train dataset에 사용되는 검사자 수: {}".format(len(train_X)))
print("Test dataset에 사용되는 검사자 수: {}".format(len(test_X)))
cancer_df = pd.DataFrame(cancer_X, columns=cancer['feature_names'])
cancer_df.head()
전체 검사자 수: 569 Attribute 수: 30 Train dataset에 사용되는 검사자 수: 512 Test dataset에 사용되는 검사자 수: 57
Out[4]:
mean radius | mean texture | mean perimeter | mean area | mean smoothness | mean compactness | mean concavity | mean concave points | mean symmetry | mean fractal dimension | ... | worst radius | worst texture | worst perimeter | worst area | worst smoothness | worst compactness | worst concavity | worst concave points | worst symmetry | worst fractal dimension | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 17.99 | 10.38 | 122.80 | 1001.0 | 0.11840 | 0.27760 | 0.3001 | 0.14710 | 0.2419 | 0.07871 | ... | 25.38 | 17.33 | 184.60 | 2019.0 | 0.1622 | 0.6656 | 0.7119 | 0.2654 | 0.4601 | 0.11890 |
1 | 20.57 | 17.77 | 132.90 | 1326.0 | 0.08474 | 0.07864 | 0.0869 | 0.07017 | 0.1812 | 0.05667 | ... | 24.99 | 23.41 | 158.80 | 1956.0 | 0.1238 | 0.1866 | 0.2416 | 0.1860 | 0.2750 | 0.08902 |
2 | 19.69 | 21.25 | 130.00 | 1203.0 | 0.10960 | 0.15990 | 0.1974 | 0.12790 | 0.2069 | 0.05999 | ... | 23.57 | 25.53 | 152.50 | 1709.0 | 0.1444 | 0.4245 | 0.4504 | 0.2430 | 0.3613 | 0.08758 |
3 | 11.42 | 20.38 | 77.58 | 386.1 | 0.14250 | 0.28390 | 0.2414 | 0.10520 | 0.2597 | 0.09744 | ... | 14.91 | 26.50 | 98.87 | 567.7 | 0.2098 | 0.8663 | 0.6869 | 0.2575 | 0.6638 | 0.17300 |
4 | 20.29 | 14.34 | 135.10 | 1297.0 | 0.10030 | 0.13280 | 0.1980 | 0.10430 | 0.1809 | 0.05883 | ... | 22.54 | 16.67 | 152.20 | 1575.0 | 0.1374 | 0.2050 | 0.4000 | 0.1625 | 0.2364 | 0.07678 |
5 rows × 30 columns
In [5]:
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
LR = LogisticRegression() #로지스틱 회귀분석
LR.fit(train_X, train_y) # 유방암 train data를 이용하여 로지스틱 회귀분석 모델 학습
pred = LR.predict(test_X) # 학습된 로지스틱 회귀분석 모델을 이용한 예측값 출력
# 로지스틱 회귀분석 모델의 예측값과 실제값 비교결과를 나타낸 통계표
print(classification_report(test_y, pred))
precision recall f1-score support 0 0.91 0.95 0.93 21 1 0.97 0.94 0.96 36 accuracy 0.95 57 macro avg 0.94 0.95 0.94 57 weighted avg 0.95 0.95 0.95 57
/home/ssac24/anaconda3/envs/aiffel/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:762: ConvergenceWarning: lbfgs failed to converge (status=1): STOP: TOTAL NO. of ITERATIONS REACHED LIMIT. Increase the number of iterations (max_iter) or scale the data as shown in: https://scikit-learn.org/stable/modules/preprocessing.html Please also refer to the documentation for alternative solver options: https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
3. Softmax 함수¶
- 2가지가 아닌 여러 범주로 분류하는 함수
- 각 범주의 확률값이 0에서 1 사이의 값이고, 또 하나의 큰 특징은 모든 범주에 해당하는 softmax의 값을 전부 더했을 때 그 합이 1이 된다
- 큰 log-odds와 작은 log-odds의 차이를 극대화시켜준다
- 마지막에 softmax 함수에 모든 범주의 log-odds를 통과시키면 해당 데이터가 어떤 범주로 분류되는지 확실히 알 수 있게 되는데, 가장 큰 값을 1, 그 외 나머지 값들을 0으로 인코딩하는 one-hot encoding을 통해 표현하게 된다
4. Cross Entropy¶
- softmax 함수의 손실함수
- 로지스틱 회귀모델이 추론한 확률 분포와 실제 데이터의 분포의 차이를 계산한 것
- Cross-entropy 의 이해: 정보이론과의 관계
In [6]:
import tensorflow.compat.v1 as tf
import numpy as np
tf.compat.v1.disable_eager_execution()
x = tf.placeholder(tf.float32, [None, 30]) # 30개의 attribute를 가진 유방암 데이터셋
y = tf.placeholder(tf.float32, [None, 2]) # 악성, 양성 label에 one-hot encoding을 적용하여 2차원이 될 예정
num_classes = 2 # 악성, 양성
lr = 0.0003 # learning rate
W = tf.Variable(tf.random_normal([30, num_classes]), name='weight') # 가중치
b = tf.Variable(tf.random_normal([num_classes]), name='bias') # bias
logits = tf.matmul(x, W) + b # log-odds에 해당되는 값
hypothesis = tf.nn.softmax(logits) # log-odds에 softmax 적용한 값
loss_i = tf.nn.softmax_cross_entropy_with_logits(
logits=logits, labels=y) # i번째 변수의 p(x)*log(q(x))
loss = tf.reduce_mean(loss_i) # 이를 모두 summation한 최종 loss function
#최적화함수로 gradient descent method 사용
optimizer = tf.train.GradientDescentOptimizer(learning_rate=lr).minimize(loss)
pred = tf.argmax(hypothesis, 1) # hypothesis의 결과 큰 값을 1, 작은 값을 0으로 매핑
correct_pred = tf.equal(pred, tf.argmax(y, 1)) # 예측값과 ground truth 값을 비교해서 같으면 True, 다르면 False
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32)) # True는 1, False는 0이 되기 때문에 전체 평균값이 결국 accuracy가 됩니다.
with tf.Session() as sess:
sess.run(tf.global_variables_initializer()) # Tensorflow의 전역 변수 초기화
one_hot_train_y = np.eye(num_classes)[train_y] # one-hot encoding을 수행한 train data 라벨
one_hot_test_y = np.eye(num_classes)[test_y] # one-hot encoding을 수행한 test data 라벨
# 전체 train data를 1000번 iteration
for step in range(1000):
sess.run(optimizer, feed_dict={x: train_X, y: one_hot_train_y}) # train data로 학습
test_loss, test_acc= sess.run([loss, accuracy], feed_dict={x: test_X, y: one_hot_test_y}) #학습된 모델을 통해 측정한 test data의 loss 값 및 정확도
if step % 10 == 0: # 10번째 step마다 print 수행
print("Step {}, loss: {:.3f} accuracy: {:.3f}".format(
step,
test_loss,
test_acc
))
Step 0, loss: 107.011 accuracy: 0.351 Step 10, loss: 152.560 accuracy: 0.368 Step 20, loss: 133.738 accuracy: 0.368 Step 30, loss: 161.270 accuracy: 0.368 Step 40, loss: 147.187 accuracy: 0.368 Step 50, loss: 141.652 accuracy: 0.368 Step 60, loss: 132.871 accuracy: 0.368 Step 70, loss: 22.773 accuracy: 0.649 Step 80, loss: 24.449 accuracy: 0.368 Step 90, loss: 113.660 accuracy: 0.368 Step 100, loss: 105.795 accuracy: 0.368 Step 110, loss: 45.783 accuracy: 0.368 Step 120, loss: 89.359 accuracy: 0.368 Step 130, loss: 48.449 accuracy: 0.368 Step 140, loss: 66.134 accuracy: 0.368 Step 150, loss: 58.432 accuracy: 0.368 Step 160, loss: 51.506 accuracy: 0.368 Step 170, loss: 45.619 accuracy: 0.386 Step 180, loss: 38.864 accuracy: 0.404 Step 190, loss: 32.437 accuracy: 0.439 Step 200, loss: 24.099 accuracy: 0.509 Step 210, loss: 17.890 accuracy: 0.596 Step 220, loss: 12.019 accuracy: 0.702 Step 230, loss: 11.274 accuracy: 0.754 Step 240, loss: 5.686 accuracy: 0.860 Step 250, loss: 5.634 accuracy: 0.860 Step 260, loss: 5.826 accuracy: 0.860 Step 270, loss: 5.808 accuracy: 0.860 Step 280, loss: 5.762 accuracy: 0.860 Step 290, loss: 5.713 accuracy: 0.860 Step 300, loss: 5.664 accuracy: 0.860 Step 310, loss: 5.619 accuracy: 0.860 Step 320, loss: 5.580 accuracy: 0.860 Step 330, loss: 5.555 accuracy: 0.860 Step 340, loss: 5.520 accuracy: 0.860 Step 350, loss: 5.467 accuracy: 0.860 Step 360, loss: 5.430 accuracy: 0.860 Step 370, loss: 5.387 accuracy: 0.860 Step 380, loss: 5.410 accuracy: 0.877 Step 390, loss: 5.408 accuracy: 0.860 Step 400, loss: 5.324 accuracy: 0.877 Step 410, loss: 5.373 accuracy: 0.860 Step 420, loss: 10.937 accuracy: 0.702 Step 430, loss: 25.104 accuracy: 0.509 Step 440, loss: 22.573 accuracy: 0.561 Step 450, loss: 18.363 accuracy: 0.632 Step 460, loss: 9.583 accuracy: 0.789 Step 470, loss: 4.887 accuracy: 0.877 Step 480, loss: 4.489 accuracy: 0.912 Step 490, loss: 4.443 accuracy: 0.895 Step 500, loss: 4.406 accuracy: 0.895 Step 510, loss: 4.371 accuracy: 0.895 Step 520, loss: 4.338 accuracy: 0.895 Step 530, loss: 4.305 accuracy: 0.895 Step 540, loss: 4.272 accuracy: 0.895 Step 550, loss: 4.239 accuracy: 0.895 Step 560, loss: 4.201 accuracy: 0.895 Step 570, loss: 4.281 accuracy: 0.895 Step 580, loss: 4.605 accuracy: 0.895 Step 590, loss: 4.744 accuracy: 0.895 Step 600, loss: 4.744 accuracy: 0.895 Step 610, loss: 4.747 accuracy: 0.895 Step 620, loss: 4.747 accuracy: 0.895 Step 630, loss: 4.744 accuracy: 0.895 Step 640, loss: 4.744 accuracy: 0.895 Step 650, loss: 4.912 accuracy: 0.895 Step 660, loss: 25.795 accuracy: 0.684 Step 670, loss: 22.608 accuracy: 0.702 Step 680, loss: 20.326 accuracy: 0.754 Step 690, loss: 12.760 accuracy: 0.807 Step 700, loss: 5.064 accuracy: 0.895 Step 710, loss: 4.870 accuracy: 0.895 Step 720, loss: 4.798 accuracy: 0.895 Step 730, loss: 4.808 accuracy: 0.895 Step 740, loss: 4.794 accuracy: 0.895 Step 750, loss: 4.784 accuracy: 0.895 Step 760, loss: 4.784 accuracy: 0.895 Step 770, loss: 4.616 accuracy: 0.895 Step 780, loss: 4.703 accuracy: 0.895 Step 790, loss: 4.598 accuracy: 0.895 Step 800, loss: 4.646 accuracy: 0.895 Step 810, loss: 4.598 accuracy: 0.895 Step 820, loss: 4.598 accuracy: 0.895 Step 830, loss: 4.595 accuracy: 0.895 Step 840, loss: 4.604 accuracy: 0.895 Step 850, loss: 4.622 accuracy: 0.895 Step 860, loss: 4.631 accuracy: 0.895 Step 870, loss: 4.624 accuracy: 0.895 Step 880, loss: 4.613 accuracy: 0.895 Step 890, loss: 4.600 accuracy: 0.895 Step 900, loss: 4.587 accuracy: 0.895 Step 910, loss: 4.573 accuracy: 0.895 Step 920, loss: 4.560 accuracy: 0.895 Step 930, loss: 4.547 accuracy: 0.895 Step 940, loss: 4.534 accuracy: 0.895 Step 950, loss: 4.522 accuracy: 0.895 Step 960, loss: 4.510 accuracy: 0.895 Step 970, loss: 4.496 accuracy: 0.895 Step 980, loss: 4.471 accuracy: 0.895 Step 990, loss: 4.421 accuracy: 0.895
스텝을 거듭할수록 정확도가 올라가는 것을 볼 수 있다.¶
In [7]:
import tensorflow as tf
from tensorflow import keras
n_dense=30
n_train_epoch=20
num_classes = 2 # 악성, 양성
model=keras.models.Sequential()
model.add(keras.layers.Dense(num_classes, use_bias=True, activation='softmax', input_shape=(30,)))
model.summary()
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
# 모델 훈련
model.fit(train_X, train_y, epochs=n_train_epoch)
# 모델 시험
test_loss, test_accuracy = model.evaluate(test_X, test_y, verbose=2)
print("test_loss: {} ".format(test_loss))
print("test_accuracy: {}".format(test_accuracy))
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense (Dense) (None, 2) 62 ================================================================= Total params: 62 Trainable params: 62 Non-trainable params: 0 _________________________________________________________________ Train on 512 samples Epoch 1/20 512/512 [==============================] - 0s 20us/sample - loss: 229.5185 - accuracy: 0.3730 Epoch 2/20 512/512 [==============================] - 0s 18us/sample - loss: 204.1008 - accuracy: 0.3730 Epoch 3/20 512/512 [==============================] - 0s 17us/sample - loss: 178.4884 - accuracy: 0.3730 Epoch 4/20 512/512 [==============================] - 0s 18us/sample - loss: 153.2470 - accuracy: 0.3730 Epoch 5/20 512/512 [==============================] - 0s 17us/sample - loss: 127.5923 - accuracy: 0.3730 Epoch 6/20 512/512 [==============================] - 0s 16us/sample - loss: 101.9865 - accuracy: 0.3730 Epoch 7/20 512/512 [==============================] - 0s 19us/sample - loss: 77.0795 - accuracy: 0.3730 Epoch 8/20 512/512 [==============================] - 0s 17us/sample - loss: 51.1562 - accuracy: 0.3730 Epoch 9/20 512/512 [==============================] - 0s 16us/sample - loss: 25.9666 - accuracy: 0.3730 Epoch 10/20 512/512 [==============================] - 0s 18us/sample - loss: 4.3849 - accuracy: 0.6289 Epoch 11/20 512/512 [==============================] - 0s 18us/sample - loss: 1.9882 - accuracy: 0.8398 Epoch 12/20 512/512 [==============================] - 0s 19us/sample - loss: 0.9919 - accuracy: 0.8398 Epoch 13/20 512/512 [==============================] - 0s 19us/sample - loss: 0.9842 - accuracy: 0.8359 Epoch 14/20 512/512 [==============================] - 0s 16us/sample - loss: 0.8851 - accuracy: 0.8730 Epoch 15/20 512/512 [==============================] - 0s 18us/sample - loss: 0.8827 - accuracy: 0.8438 Epoch 16/20 512/512 [==============================] - 0s 17us/sample - loss: 0.8789 - accuracy: 0.8613 Epoch 17/20 512/512 [==============================] - 0s 17us/sample - loss: 0.8448 - accuracy: 0.8535 Epoch 18/20 512/512 [==============================] - 0s 18us/sample - loss: 0.8449 - accuracy: 0.8477 Epoch 19/20 512/512 [==============================] - 0s 18us/sample - loss: 0.8290 - accuracy: 0.8672 Epoch 20/20 512/512 [==============================] - 0s 17us/sample - loss: 0.8169 - accuracy: 0.8535 test_loss: 0.9484101931254069 test_accuracy: 0.8245614171028137
In [8]:
import tensorflow as tf
from tensorflow import keras
n_dense=30
n_train_epoch=20
num_classes = 2 # 악성, 양성
model=keras.models.Sequential()
#레이어 3장을 추가
model.add(keras.layers.Dense(n_dense, input_shape=(30,), use_bias=True))
model.add(keras.layers.Dense(n_dense, use_bias=True))
model.add(keras.layers.Dense(n_dense, use_bias=True))
model.add(keras.layers.Dense(num_classes, use_bias=True, activation='softmax'))
model.summary()
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
# 모델 훈련
model.fit(train_X, train_y, epochs=n_train_epoch)
# 모델 시험
test_loss, test_accuracy = model.evaluate(test_X, test_y, verbose=2)
print("test_loss: {} ".format(test_loss))
print("test_accuracy: {}".format(test_accuracy))
Model: "sequential_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense_1 (Dense) (None, 30) 930 _________________________________________________________________ dense_2 (Dense) (None, 30) 930 _________________________________________________________________ dense_3 (Dense) (None, 30) 930 _________________________________________________________________ dense_4 (Dense) (None, 2) 62 ================================================================= Total params: 2,852 Trainable params: 2,852 Non-trainable params: 0 _________________________________________________________________ Train on 512 samples Epoch 1/20 512/512 [==============================] - 0s 22us/sample - loss: 29.8753 - accuracy: 0.5020 Epoch 2/20 512/512 [==============================] - 0s 22us/sample - loss: 4.7048 - accuracy: 0.7051 Epoch 3/20 512/512 [==============================] - 0s 23us/sample - loss: 2.3118 - accuracy: 0.8438 Epoch 4/20 512/512 [==============================] - 0s 21us/sample - loss: 2.2221 - accuracy: 0.8496 Epoch 5/20 512/512 [==============================] - 0s 22us/sample - loss: 1.6885 - accuracy: 0.8926 Epoch 6/20 512/512 [==============================] - 0s 23us/sample - loss: 2.0578 - accuracy: 0.8594 Epoch 7/20 512/512 [==============================] - 0s 25us/sample - loss: 1.9662 - accuracy: 0.8418 Epoch 8/20 512/512 [==============================] - 0s 24us/sample - loss: 1.4209 - accuracy: 0.8730 Epoch 9/20 512/512 [==============================] - 0s 22us/sample - loss: 1.6398 - accuracy: 0.8730 Epoch 10/20 512/512 [==============================] - 0s 23us/sample - loss: 1.7562 - accuracy: 0.8672 Epoch 11/20 512/512 [==============================] - 0s 21us/sample - loss: 1.4389 - accuracy: 0.8809 Epoch 12/20 512/512 [==============================] - 0s 20us/sample - loss: 1.0506 - accuracy: 0.8984 Epoch 13/20 512/512 [==============================] - 0s 21us/sample - loss: 1.1184 - accuracy: 0.8945 Epoch 14/20 512/512 [==============================] - 0s 22us/sample - loss: 1.0091 - accuracy: 0.9062 Epoch 15/20 512/512 [==============================] - 0s 23us/sample - loss: 1.2391 - accuracy: 0.8633 Epoch 16/20 512/512 [==============================] - 0s 20us/sample - loss: 1.0095 - accuracy: 0.9062 Epoch 17/20 512/512 [==============================] - 0s 22us/sample - loss: 0.7776 - accuracy: 0.9062 Epoch 18/20 512/512 [==============================] - 0s 20us/sample - loss: 0.8810 - accuracy: 0.9023 Epoch 19/20 512/512 [==============================] - 0s 23us/sample - loss: 1.4091 - accuracy: 0.8574 Epoch 20/20 512/512 [==============================] - 0s 22us/sample - loss: 0.9340 - accuracy: 0.8984 test_loss: 3.2419562507094 test_accuracy: 0.719298243522644
데이터셋 자체가 적어 레이어를 나누는 것에 큰 차이가 없음을 알 수 있다.¶
5. 정리¶
'파이썬 & AI 학습' 카테고리의 다른 글
이상치 탐색 (0) | 2021.02.19 |
---|---|
데이터 가져오기 (0) | 2021.02.15 |
데이터 전처리 (0) | 2021.02.03 |
파이썬 문법 (0) | 2021.02.01 |
비지도 학습( Unsupervised Learning ) (0) | 2021.01.29 |