선형회귀와 로지스틱회귀

회귀(Regression)¶

---

1. 선형 회귀 분석(Linear Regression)¶

• 종속변수 Y와 한 개 이상의 독립변수 X와의 선형 상관관계를 모델링하는 회귀분석 기법
• 주어진 데이터에 우리의 선형 식이 잘 맞도록 회귀계수 및 오차를 구하는 것
• 머신러닝에서의 선형회귀모델 H = Wx + b
  가정(Hypothesis), 가중치(Weight), 편향(bias)
• 손실함수 중 최소제곱법이란 n 개의 점 데이터에 대하여 잔차의 제곱의 합을 최소로 하는 W, b를 구하는 방법
  • 잔차란 회귀모델을 이용해 추정한 값과 실제 데이터의 차이를 뜻함.
• 회귀모델이 잘 결정되었는지 R2-Score를 통해 알 수 있음(1에 가까울수록 회귀모델이 데이터를 잘 표현하고 있음)

1-1. 최소제곱법¶

In [2]:

# boston house prices dataset에 대한 정보
from sklearn.datasets import load_boston
from sklearn import model_selection
import pandas as pd
import numpy as np

# 데이터 로드
boston = load_boston()
data, price = boston['data'], boston['target']
x_train, x_test, y_train, y_test = model_selection.train_test_split(data, price, test_size=0.2)

df = pd.DataFrame(x_train, columns=boston['feature_names'])
print("boston dataset의 차원: ", data.shape)
print("price의 차원", price.shape)
print("boston train dataset의 차원: ", x_train.shape)
print("boston test dataset의 차원: ", x_test.shape)

print("Boston dataset의 예시")
df.head() # data 요약

boston dataset의 차원:  (506, 13)
price의 차원 (506,)
boston train dataset의 차원:  (404, 13)
boston test dataset의 차원:  (102, 13)
Boston dataset의 예시

Out[2]:

	CRIM	ZN	INDUS	CHAS	NOX	RM	AGE	DIS	RAD	TAX	PTRATIO	B	LSTAT
0	25.04610	0.0	18.10	0.0	0.693	5.987	100.0	1.5888	24.0	666.0	20.2	396.90	26.77
1	0.04379	80.0	3.37	0.0	0.398	5.787	31.1	6.6115	4.0	337.0	16.1	396.90	10.24
2	15.86030	0.0	18.10	0.0	0.679	5.896	95.4	1.9096	24.0	666.0	20.2	7.68	24.39
3	0.21719	0.0	10.59	1.0	0.489	5.807	53.8	3.6526	4.0	277.0	18.6	390.94	16.03
4	4.42228	0.0	18.10	0.0	0.584	6.003	94.5	2.5403	24.0	666.0	20.2	331.29	21.32

In [19]:

print(boston["DESCR"])

.. _boston_dataset:

Boston house prices dataset
---------------------------

**Data Set Characteristics:**  

    :Number of Instances: 506 

    :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target.

    :Attribute Information (in order):
        - CRIM     per capita crime rate by town
        - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.
        - INDUS    proportion of non-retail business acres per town
        - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
        - NOX      nitric oxides concentration (parts per 10 million)
        - RM       average number of rooms per dwelling
        - AGE      proportion of owner-occupied units built prior to 1940
        - DIS      weighted distances to five Boston employment centres
        - RAD      index of accessibility to radial highways
        - TAX      full-value property-tax rate per $10,000
        - PTRATIO  pupil-teacher ratio by town
        - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town
        - LSTAT    % lower status of the population
        - MEDV     Median value of owner-occupied homes in $1000's

    :Missing Attribute Values: None

    :Creator: Harrison, D. and Rubinfeld, D.L.

This is a copy of UCI ML housing dataset.
https://archive.ics.uci.edu/ml/machine-learning-databases/housing/


This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.

The Boston house-price data of Harrison, D. and Rubinfeld, D.L. 'Hedonic
prices and the demand for clean air', J. Environ. Economics & Management,
vol.5, 81-102, 1978.   Used in Belsley, Kuh & Welsch, 'Regression diagnostics
...', Wiley, 1980.   N.B. Various transformations are used in the table on
pages 244-261 of the latter.

The Boston house-price data has been used in many machine learning papers that address regression
problems.   
     
.. topic:: References

   - Belsley, Kuh & Welsch, 'Regression diagnostics: Identifying Influential Data and Sources of Collinearity', Wiley, 1980. 244-261.
   - Quinlan,R. (1993). Combining Instance-Based and Model-Based Learning. In Proceedings on the Tenth International Conference of Machine Learning, 236-243, University of Massachusetts, Amherst. Morgan Kaufmann.

In [20]:

# Boston dataset의 각 attribute에 선형회귀 적용하는 예제

import pandas as pd
from sklearn import datasets
from sklearn import model_selection
from sklearn.linear_model import LinearRegression
from sklearn import metrics
from sklearn import datasets
import matplotlib.pyplot as plt

fig = plt.figure(figsize=(10,35))
fig.suptitle('Boston dataset - (X:Y = each attr: price) with R2', fontsize=16, y=0.9)

for i in range(data.shape[1]): # boston dataset에서 i번째 attribute(column)을 살펴볼 거에요.

    single_attr, attr_name = data[:, i].reshape(-1, 1), boston['feature_names'][i] # i번째 attribute에 대한 data 및 이름
    estimator = LinearRegression() # 선형 회귀 모델이에요.

    #x에는 single_attr, y에는 price에 해당하는 데이터를 대입해서 최소제곱법을 이용하여 모델 내에서 W, b를 구하는 과정이에요
    estimator.fit(single_attr, price) 

    #위 fit() 과정을 통해 구한 회귀계수를 기반으로 회귀모델에 X값을 대입했을 때의 예측 Y 값이에요. 
    pred_price = estimator.predict(single_attr)

    score = metrics.r2_score(price, pred_price) # 결정계수를 구하는 함수에요. 

    # 캔버스 생성
    ax = fig.add_subplot(7, 2, i+1)
    ax.scatter(single_attr, price) # 실제 데이터에 대한 산포도
    ax.plot(single_attr, pred_price, color='red') # 선형회귀모델의 추세선
    ax.set_title("{} x price, R2 score={:.3f}".format(attr_name ,score)) #subplot의 제목이에요
    ax.set_xlabel(attr_name) # x축
    ax.set_ylabel('price') # y축

1-2. 경사 하강법(Gradient Descent Algorithm)¶

• 경사 하강의 최소지점이 손실함수의 최소값을 나타낸다고 생각하여 최적의 그래디언트값을 찾아 최적의 W값을 찾는 방법

2. 로지스틱 회귀분석(Logistic Regression)¶

• [0, 1]사이의 값으로 나타내기때문에 1개 이상의 독립변수가 있을 때 이를 이용하여 데이터가 2개의 범주 중 하나에 속하도록 결정하는 이진 분류(binary classification) 많이 사용
• sigmoid function의 순서

1. 실제 데이터를 대입하여 Odds 및 회귀계수를 구한다.
2. Log-odds를 계산한 후, 이를 sigmoid function의 입력으로 넣어서 특정 범주에 속할 확률값을 계산한다.
3. 설정한 threshold에 맞추어 설정값 이상이면 1, 이하면 0으로 이진분류를 수행한다.

In [4]:

# 로지스틱 회귀분석 예제: 유방암 데이터셋
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

# 데이터 로드
cancer=load_breast_cancer()

# y = 0(Malignant - 악성 종양), y=1(Benign - 양성 종양)
cancer_X, cancer_y= cancer.data, cancer['target']
train_X, test_X, train_y, test_y = train_test_split(cancer_X, cancer_y, test_size=0.1, random_state=10) # train 데이터셋과 test 데이터셋으로 나눔
print("전체 검사자 수: {}".format(len(cancer_X)))
print("Attribute 수: {}".format(len(cancer_X[0])))
print("Train dataset에 사용되는 검사자 수: {}".format(len(train_X)))
print("Test dataset에 사용되는 검사자 수: {}".format(len(test_X)))
cancer_df = pd.DataFrame(cancer_X, columns=cancer['feature_names'])
cancer_df.head()

전체 검사자 수: 569
Attribute 수: 30
Train dataset에 사용되는 검사자 수: 512
Test dataset에 사용되는 검사자 수: 57

Out[4]:

	mean radius	mean texture	mean perimeter	mean area	mean smoothness	mean compactness	mean concavity	mean concave points	mean symmetry	mean fractal dimension	...	worst radius	worst texture	worst perimeter	worst area	worst smoothness	worst compactness	worst concavity	worst concave points	worst symmetry	worst fractal dimension
0	17.99	10.38	122.80	1001.0	0.11840	0.27760	0.3001	0.14710	0.2419	0.07871	...	25.38	17.33	184.60	2019.0	0.1622	0.6656	0.7119	0.2654	0.4601	0.11890
1	20.57	17.77	132.90	1326.0	0.08474	0.07864	0.0869	0.07017	0.1812	0.05667	...	24.99	23.41	158.80	1956.0	0.1238	0.1866	0.2416	0.1860	0.2750	0.08902
2	19.69	21.25	130.00	1203.0	0.10960	0.15990	0.1974	0.12790	0.2069	0.05999	...	23.57	25.53	152.50	1709.0	0.1444	0.4245	0.4504	0.2430	0.3613	0.08758
3	11.42	20.38	77.58	386.1	0.14250	0.28390	0.2414	0.10520	0.2597	0.09744	...	14.91	26.50	98.87	567.7	0.2098	0.8663	0.6869	0.2575	0.6638	0.17300
4	20.29	14.34	135.10	1297.0	0.10030	0.13280	0.1980	0.10430	0.1809	0.05883	...	22.54	16.67	152.20	1575.0	0.1374	0.2050	0.4000	0.1625	0.2364	0.07678

5 rows × 30 columns

In [5]:

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report

LR = LogisticRegression() #로지스틱 회귀분석
LR.fit(train_X, train_y) # 유방암 train data를 이용하여 로지스틱 회귀분석 모델 학습
pred = LR.predict(test_X) # 학습된 로지스틱 회귀분석 모델을 이용한 예측값 출력


# 로지스틱 회귀분석 모델의 예측값과 실제값 비교결과를 나타낸 통계표
print(classification_report(test_y, pred))

              precision    recall  f1-score   support

           0       0.91      0.95      0.93        21
           1       0.97      0.94      0.96        36

    accuracy                           0.95        57
   macro avg       0.94      0.95      0.94        57
weighted avg       0.95      0.95      0.95        57

/home/ssac24/anaconda3/envs/aiffel/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:762: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
    https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)

3. Softmax 함수¶

• 2가지가 아닌 여러 범주로 분류하는 함수
  
• 각 범주의 확률값이 0에서 1 사이의 값이고, 또 하나의 큰 특징은 모든 범주에 해당하는 softmax의 값을 전부 더했을 때 그 합이 1이 된다
• 큰 log-odds와 작은 log-odds의 차이를 극대화시켜준다
• 마지막에 softmax 함수에 모든 범주의 log-odds를 통과시키면 해당 데이터가 어떤 범주로 분류되는지 확실히 알 수 있게 되는데, 가장 큰 값을 1, 그 외 나머지 값들을 0으로 인코딩하는 one-hot encoding을 통해 표현하게 된다

4. Cross Entropy¶

• softmax 함수의 손실함수
• 로지스틱 회귀모델이 추론한 확률 분포와 실제 데이터의 분포의 차이를 계산한 것
• Cross-entropy 의 이해: 정보이론과의 관계

In [6]:

import tensorflow.compat.v1 as tf
import numpy as np
tf.compat.v1.disable_eager_execution()

x = tf.placeholder(tf.float32, [None, 30]) # 30개의 attribute를 가진 유방암 데이터셋
y = tf.placeholder(tf.float32, [None, 2]) # 악성, 양성 label에 one-hot encoding을 적용하여 2차원이 될 예정 
num_classes = 2 # 악성, 양성
lr = 0.0003 # learning rate

W = tf.Variable(tf.random_normal([30, num_classes]), name='weight') # 가중치
b = tf.Variable(tf.random_normal([num_classes]), name='bias') # bias

logits = tf.matmul(x, W) + b # log-odds에 해당되는 값
hypothesis = tf.nn.softmax(logits) # log-odds에 softmax 적용한 값

loss_i = tf.nn.softmax_cross_entropy_with_logits(
    logits=logits, labels=y) # i번째 변수의 p(x)*log(q(x))
loss = tf.reduce_mean(loss_i) # 이를 모두 summation한 최종 loss function

#최적화함수로 gradient descent method 사용
optimizer = tf.train.GradientDescentOptimizer(learning_rate=lr).minimize(loss)

pred = tf.argmax(hypothesis, 1) # hypothesis의 결과 큰 값을 1, 작은 값을 0으로 매핑
correct_pred = tf.equal(pred, tf.argmax(y, 1)) # 예측값과 ground truth 값을 비교해서 같으면 True, 다르면 False
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32)) # True는 1, False는 0이 되기 때문에 전체 평균값이 결국 accuracy가 됩니다.


with tf.Session() as sess:
    sess.run(tf.global_variables_initializer()) # Tensorflow의 전역 변수 초기화
    one_hot_train_y = np.eye(num_classes)[train_y] # one-hot encoding을 수행한 train data 라벨
    one_hot_test_y = np.eye(num_classes)[test_y] # one-hot encoding을 수행한 test data 라벨
    
    # 전체 train data를 1000번 iteration
    for step in range(1000):
        sess.run(optimizer, feed_dict={x: train_X, y: one_hot_train_y}) # train data로 학습
        test_loss, test_acc= sess.run([loss, accuracy], feed_dict={x: test_X, y: one_hot_test_y}) #학습된 모델을 통해 측정한 test data의 loss 값 및 정확도
        if step % 10 == 0: # 10번째 step마다 print 수행
            print("Step {}, loss: {:.3f} accuracy: {:.3f}".format(
            step, 
            test_loss,
            test_acc
            ))

Step 0, loss: 107.011 accuracy: 0.351
Step 10, loss: 152.560 accuracy: 0.368
Step 20, loss: 133.738 accuracy: 0.368
Step 30, loss: 161.270 accuracy: 0.368
Step 40, loss: 147.187 accuracy: 0.368
Step 50, loss: 141.652 accuracy: 0.368
Step 60, loss: 132.871 accuracy: 0.368
Step 70, loss: 22.773 accuracy: 0.649
Step 80, loss: 24.449 accuracy: 0.368
Step 90, loss: 113.660 accuracy: 0.368
Step 100, loss: 105.795 accuracy: 0.368
Step 110, loss: 45.783 accuracy: 0.368
Step 120, loss: 89.359 accuracy: 0.368
Step 130, loss: 48.449 accuracy: 0.368
Step 140, loss: 66.134 accuracy: 0.368
Step 150, loss: 58.432 accuracy: 0.368
Step 160, loss: 51.506 accuracy: 0.368
Step 170, loss: 45.619 accuracy: 0.386
Step 180, loss: 38.864 accuracy: 0.404
Step 190, loss: 32.437 accuracy: 0.439
Step 200, loss: 24.099 accuracy: 0.509
Step 210, loss: 17.890 accuracy: 0.596
Step 220, loss: 12.019 accuracy: 0.702
Step 230, loss: 11.274 accuracy: 0.754
Step 240, loss: 5.686 accuracy: 0.860
Step 250, loss: 5.634 accuracy: 0.860
Step 260, loss: 5.826 accuracy: 0.860
Step 270, loss: 5.808 accuracy: 0.860
Step 280, loss: 5.762 accuracy: 0.860
Step 290, loss: 5.713 accuracy: 0.860
Step 300, loss: 5.664 accuracy: 0.860
Step 310, loss: 5.619 accuracy: 0.860
Step 320, loss: 5.580 accuracy: 0.860
Step 330, loss: 5.555 accuracy: 0.860
Step 340, loss: 5.520 accuracy: 0.860
Step 350, loss: 5.467 accuracy: 0.860
Step 360, loss: 5.430 accuracy: 0.860
Step 370, loss: 5.387 accuracy: 0.860
Step 380, loss: 5.410 accuracy: 0.877
Step 390, loss: 5.408 accuracy: 0.860
Step 400, loss: 5.324 accuracy: 0.877
Step 410, loss: 5.373 accuracy: 0.860
Step 420, loss: 10.937 accuracy: 0.702
Step 430, loss: 25.104 accuracy: 0.509
Step 440, loss: 22.573 accuracy: 0.561
Step 450, loss: 18.363 accuracy: 0.632
Step 460, loss: 9.583 accuracy: 0.789
Step 470, loss: 4.887 accuracy: 0.877
Step 480, loss: 4.489 accuracy: 0.912
Step 490, loss: 4.443 accuracy: 0.895
Step 500, loss: 4.406 accuracy: 0.895
Step 510, loss: 4.371 accuracy: 0.895
Step 520, loss: 4.338 accuracy: 0.895
Step 530, loss: 4.305 accuracy: 0.895
Step 540, loss: 4.272 accuracy: 0.895
Step 550, loss: 4.239 accuracy: 0.895
Step 560, loss: 4.201 accuracy: 0.895
Step 570, loss: 4.281 accuracy: 0.895
Step 580, loss: 4.605 accuracy: 0.895
Step 590, loss: 4.744 accuracy: 0.895
Step 600, loss: 4.744 accuracy: 0.895
Step 610, loss: 4.747 accuracy: 0.895
Step 620, loss: 4.747 accuracy: 0.895
Step 630, loss: 4.744 accuracy: 0.895
Step 640, loss: 4.744 accuracy: 0.895
Step 650, loss: 4.912 accuracy: 0.895
Step 660, loss: 25.795 accuracy: 0.684
Step 670, loss: 22.608 accuracy: 0.702
Step 680, loss: 20.326 accuracy: 0.754
Step 690, loss: 12.760 accuracy: 0.807
Step 700, loss: 5.064 accuracy: 0.895
Step 710, loss: 4.870 accuracy: 0.895
Step 720, loss: 4.798 accuracy: 0.895
Step 730, loss: 4.808 accuracy: 0.895
Step 740, loss: 4.794 accuracy: 0.895
Step 750, loss: 4.784 accuracy: 0.895
Step 760, loss: 4.784 accuracy: 0.895
Step 770, loss: 4.616 accuracy: 0.895
Step 780, loss: 4.703 accuracy: 0.895
Step 790, loss: 4.598 accuracy: 0.895
Step 800, loss: 4.646 accuracy: 0.895
Step 810, loss: 4.598 accuracy: 0.895
Step 820, loss: 4.598 accuracy: 0.895
Step 830, loss: 4.595 accuracy: 0.895
Step 840, loss: 4.604 accuracy: 0.895
Step 850, loss: 4.622 accuracy: 0.895
Step 860, loss: 4.631 accuracy: 0.895
Step 870, loss: 4.624 accuracy: 0.895
Step 880, loss: 4.613 accuracy: 0.895
Step 890, loss: 4.600 accuracy: 0.895
Step 900, loss: 4.587 accuracy: 0.895
Step 910, loss: 4.573 accuracy: 0.895
Step 920, loss: 4.560 accuracy: 0.895
Step 930, loss: 4.547 accuracy: 0.895
Step 940, loss: 4.534 accuracy: 0.895
Step 950, loss: 4.522 accuracy: 0.895
Step 960, loss: 4.510 accuracy: 0.895
Step 970, loss: 4.496 accuracy: 0.895
Step 980, loss: 4.471 accuracy: 0.895
Step 990, loss: 4.421 accuracy: 0.895

스텝을 거듭할수록 정확도가 올라가는 것을 볼 수 있다.¶

In [7]:

import tensorflow as tf
from tensorflow import keras

n_dense=30
n_train_epoch=20
num_classes = 2 # 악성, 양성

model=keras.models.Sequential()
model.add(keras.layers.Dense(num_classes, use_bias=True, activation='softmax', input_shape=(30,)))

model.summary()
model.compile(optimizer='adam',
             loss='sparse_categorical_crossentropy',
             metrics=['accuracy'])

# 모델 훈련
model.fit(train_X, train_y, epochs=n_train_epoch)

# 모델 시험
test_loss, test_accuracy = model.evaluate(test_X, test_y, verbose=2)
print("test_loss: {} ".format(test_loss))
print("test_accuracy: {}".format(test_accuracy))

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 2)                 62        
=================================================================
Total params: 62
Trainable params: 62
Non-trainable params: 0
_________________________________________________________________
Train on 512 samples
Epoch 1/20
512/512 [==============================] - 0s 20us/sample - loss: 229.5185 - accuracy: 0.3730
Epoch 2/20
512/512 [==============================] - 0s 18us/sample - loss: 204.1008 - accuracy: 0.3730
Epoch 3/20
512/512 [==============================] - 0s 17us/sample - loss: 178.4884 - accuracy: 0.3730
Epoch 4/20
512/512 [==============================] - 0s 18us/sample - loss: 153.2470 - accuracy: 0.3730
Epoch 5/20
512/512 [==============================] - 0s 17us/sample - loss: 127.5923 - accuracy: 0.3730
Epoch 6/20
512/512 [==============================] - 0s 16us/sample - loss: 101.9865 - accuracy: 0.3730
Epoch 7/20
512/512 [==============================] - 0s 19us/sample - loss: 77.0795 - accuracy: 0.3730
Epoch 8/20
512/512 [==============================] - 0s 17us/sample - loss: 51.1562 - accuracy: 0.3730
Epoch 9/20
512/512 [==============================] - 0s 16us/sample - loss: 25.9666 - accuracy: 0.3730
Epoch 10/20
512/512 [==============================] - 0s 18us/sample - loss: 4.3849 - accuracy: 0.6289
Epoch 11/20
512/512 [==============================] - 0s 18us/sample - loss: 1.9882 - accuracy: 0.8398
Epoch 12/20
512/512 [==============================] - 0s 19us/sample - loss: 0.9919 - accuracy: 0.8398
Epoch 13/20
512/512 [==============================] - 0s 19us/sample - loss: 0.9842 - accuracy: 0.8359
Epoch 14/20
512/512 [==============================] - 0s 16us/sample - loss: 0.8851 - accuracy: 0.8730
Epoch 15/20
512/512 [==============================] - 0s 18us/sample - loss: 0.8827 - accuracy: 0.8438
Epoch 16/20
512/512 [==============================] - 0s 17us/sample - loss: 0.8789 - accuracy: 0.8613
Epoch 17/20
512/512 [==============================] - 0s 17us/sample - loss: 0.8448 - accuracy: 0.8535
Epoch 18/20
512/512 [==============================] - 0s 18us/sample - loss: 0.8449 - accuracy: 0.8477
Epoch 19/20
512/512 [==============================] - 0s 18us/sample - loss: 0.8290 - accuracy: 0.8672
Epoch 20/20
512/512 [==============================] - 0s 17us/sample - loss: 0.8169 - accuracy: 0.8535
test_loss: 0.9484101931254069 
test_accuracy: 0.8245614171028137

In [8]:

import tensorflow as tf
from tensorflow import keras

n_dense=30
n_train_epoch=20
num_classes = 2 # 악성, 양성

model=keras.models.Sequential()

#레이어 3장을 추가
model.add(keras.layers.Dense(n_dense, input_shape=(30,), use_bias=True))
model.add(keras.layers.Dense(n_dense,  use_bias=True))
model.add(keras.layers.Dense(n_dense,  use_bias=True))

model.add(keras.layers.Dense(num_classes, use_bias=True, activation='softmax'))

model.summary()
model.compile(optimizer='adam',
             loss='sparse_categorical_crossentropy',
             metrics=['accuracy'])

# 모델 훈련
model.fit(train_X, train_y, epochs=n_train_epoch)

# 모델 시험
test_loss, test_accuracy = model.evaluate(test_X, test_y, verbose=2)
print("test_loss: {} ".format(test_loss))
print("test_accuracy: {}".format(test_accuracy))

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 30)                930       
_________________________________________________________________
dense_2 (Dense)              (None, 30)                930       
_________________________________________________________________
dense_3 (Dense)              (None, 30)                930       
_________________________________________________________________
dense_4 (Dense)              (None, 2)                 62        
=================================================================
Total params: 2,852
Trainable params: 2,852
Non-trainable params: 0
_________________________________________________________________
Train on 512 samples
Epoch 1/20
512/512 [==============================] - 0s 22us/sample - loss: 29.8753 - accuracy: 0.5020
Epoch 2/20
512/512 [==============================] - 0s 22us/sample - loss: 4.7048 - accuracy: 0.7051
Epoch 3/20
512/512 [==============================] - 0s 23us/sample - loss: 2.3118 - accuracy: 0.8438
Epoch 4/20
512/512 [==============================] - 0s 21us/sample - loss: 2.2221 - accuracy: 0.8496
Epoch 5/20
512/512 [==============================] - 0s 22us/sample - loss: 1.6885 - accuracy: 0.8926
Epoch 6/20
512/512 [==============================] - 0s 23us/sample - loss: 2.0578 - accuracy: 0.8594
Epoch 7/20
512/512 [==============================] - 0s 25us/sample - loss: 1.9662 - accuracy: 0.8418
Epoch 8/20
512/512 [==============================] - 0s 24us/sample - loss: 1.4209 - accuracy: 0.8730
Epoch 9/20
512/512 [==============================] - 0s 22us/sample - loss: 1.6398 - accuracy: 0.8730
Epoch 10/20
512/512 [==============================] - 0s 23us/sample - loss: 1.7562 - accuracy: 0.8672
Epoch 11/20
512/512 [==============================] - 0s 21us/sample - loss: 1.4389 - accuracy: 0.8809
Epoch 12/20
512/512 [==============================] - 0s 20us/sample - loss: 1.0506 - accuracy: 0.8984
Epoch 13/20
512/512 [==============================] - 0s 21us/sample - loss: 1.1184 - accuracy: 0.8945
Epoch 14/20
512/512 [==============================] - 0s 22us/sample - loss: 1.0091 - accuracy: 0.9062
Epoch 15/20
512/512 [==============================] - 0s 23us/sample - loss: 1.2391 - accuracy: 0.8633
Epoch 16/20
512/512 [==============================] - 0s 20us/sample - loss: 1.0095 - accuracy: 0.9062
Epoch 17/20
512/512 [==============================] - 0s 22us/sample - loss: 0.7776 - accuracy: 0.9062
Epoch 18/20
512/512 [==============================] - 0s 20us/sample - loss: 0.8810 - accuracy: 0.9023
Epoch 19/20
512/512 [==============================] - 0s 23us/sample - loss: 1.4091 - accuracy: 0.8574
Epoch 20/20
512/512 [==============================] - 0s 22us/sample - loss: 0.9340 - accuracy: 0.8984
test_loss: 3.2419562507094 
test_accuracy: 0.719298243522644

데이터셋 자체가 적어 레이어를 나누는 것에 큰 차이가 없음을 알 수 있다.¶

5. 정리¶