RestServer에서 2개 이상의 모델 호출하는 방법

작성자 김아름 수정일 2022-12-07 14:33

#SparklingSoDA4.0, #스파클링소다4.0, #rest, #server, #app

아티클 관련 제품: SparklingSoDA4.0

들어가며

개발이 완료된 여러 모델을 하나의 rest server에서 호출하여 사용할 수 있습니다.
단, 해당 모델들은 input 데이터가 동일하게 사용될 때 입니다.

Model Service 생성하기

개발하여 export가 완료된 2개의 모델이 있습니다.

Model Services 탭에서 Create 합니다.

Model Service 이름과 Resource를 선택한 후, Model은 서빙하고자 하는 모델들을 선택합니다.

Status가 Running임을 확인합니다.

Rest App 생성하기

model 서비스를 호출하여 결과를 return 받을 수 있도 rest app을 생성합니다.

Rest App Name을 입력하고 Resource를 선택합니다.

생성한 rest app의 IDE를 클릭하여 노트북을 엽니다.

app_config.yaml 파일의 내용을 작성합니다.

mode: test
log_level: INFO
model_service: "mnist-svc"
model_name: "mnist"
model_version: ""

services.py 파일의 내용을 작성합니다.

import numpy as np
from app.config import cfg

# SoDAFlow Model Servce
from app.common.model_service import SodaModelService

def predict(param=None):
    # configs/app_config.yaml - model connection
    # 1번 모델 호출
    model_service = SodaModelService(
        server_url=cfg.model_service,
        model_name=cfg.model_name,
        model_version='1'
    )

    # input setting
    input_data = np.reshape(eval(param), (1,1,28,28)).astype(np.float32)

    # model input, output setting
    model_service.add_input(name='0', shape=(1,1,28,28), dtype='FP32', tensor=input_data)
    model_service.add_output(name='27')

    # inference
    results = model_service.predict()


    # 3번 모델 호출
    model_service2 = SodaModelService(
        server_url=cfg.model_service,
        model_name=cfg.model_name,
        model_version='3'
    )

    input_data = np.reshape(eval(param), (1,1,28,28)).astype(np.float32)

    model_service2.add_input(name='0', shape=(1,1,28,28), dtype='FP32', tensor=input_data)
    model_service2.add_output(name='27')

    results2 = model_service2.predict()

    return { 'predict': results, 'predict2': results2 }

controllers.py 파일의 내용을 작성합니다.

import numpy as np
import json

from flask import Blueprint, request, jsonify
from app.mod_service import services
from app.common.utils import as_json

# Define the blueprint: 'train', set its url prefix: app.url/test
mod_service = Blueprint('service', __name__, url_prefix='/api')

@mod_service.route('/predict/', methods=['GET', 'POST'])
@as_json
def predict():
    try:
        output = None

        if request.method == 'GET':
            if len(request.args.to_dict()) == 0:
                return "no parameter(get)"
            else:
                # get type parameter
                param = request.args["image_data"] 
        else:
            param = request.get_json()
            
            if param == None:
                return "no parameter(post)"
            
            else:
                # post type(json) parameter
                param = param['image_data']               

        # call inference(services.py/predict)
        output = services.predict(param)

        # data post-processing.
        post_data = output['predict'][0]['27']
        post_data2 = output['predict2'][0]['27']

        # json to array
        result = json.loads(post_data)
        result2 = json.loads(post_data2)
        
        # return
        print("1번 결과")
        print(np.argmax(result))
        print("2번 결과")
        print(np.argmax(result))

        return np.argmax(result)

    except Exception:
        result = dict()
        result['success'] = False
        result['log'] = traceback.format_exc()

        return result

local test를 위해 test_run.py와 test_clinet.sh 파일을 작성합니다.

test_run.py

from app import app

# Run a test server
app.run(host='0.0.0.0', port=8088, debug=True)

test_clinet.sh

# label 4
curl -X POST http://localhost:8088/api/predict/ -H "Content-Type: application/json" \
    -d '{"image_data":"0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,22,192,134,32,0,0,0,0,0,0,0,0,15,77,5,0,0,0,0,0,0,0,0,0,0,0,0,17,235,250,169,0,0,0,0,0,0,0,0,15,220,241,37,0,0,0,0,0,0,0,0,0,0,0,20,189,253,147,0,0,0,0,0,0,0,0,0,139,253,100,0,0,0,0,0,0,0,0,0,0,0,0,70,253,253,21,0,0,0,0,0,0,0,0,43,254,173,13,0,0,0,0,0,0,0,0,0,0,0,22,153,253,96,0,0,0,0,0,0,0,0,43,231,254,92,0,0,0,0,0,0,0,0,0,0,0,0,163,255,204,11,0,0,0,0,0,0,0,0,104,254,158,0,0,0,0,0,0,0,0,0,0,0,0,0,162,253,178,5,0,0,0,0,0,0,9,131,237,253,0,0,0,0,0,0,0,0,0,0,0,0,0,0,162,253,253,191,175,70,70,70,70,133,197,253,253,169,0,0,0,0,0,0,0,0,0,0,0,0,0,0,51,228,253,253,254,253,253,253,253,254,253,253,219,35,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,17,65,137,254,232,137,137,137,44,253,253,161,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,34,254,206,21,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,160,253,69,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,85,254,241,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,158,254,165,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,231,244,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,104,254,232,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,208,253,157,0,13,30,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,208,253,154,91,204,161,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,208,253,254,253,154,29,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,61,190,128,23,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"}'

노트북의 터미널을 열어, local test를 진행합니다.

python test_run.py 
configuration file loaded /app/mnist-restapp/app
 * Serving Flask app 'app' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Running on all addresses.
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on http://6.2.171.125:8088/ (Press CTRL+C to quit)
 * Restarting with stat
configuration file loaded /app/mnist-restapp/app
 * Debugger is active!
 * Debugger PIN: 171-405-269

터미널 창을 하나 더 열어, test_client.sh 를 실행합니다.

bash test_client.sh 
"4"

test_clinet.sh 을 실행하자, 4가 return 되는 것을 확인할 수 있고, test_run.py가 실행 중인 터미널 창에서는 아래와 같이 2개의 모델을 결과를 확인할 수 있습니다.

1번 결과
4
2번 결과
4
127.0.0.1 - - [06/Dec/2022 23:20:58] "POST /api/predict/ HTTP/1.1" 200 -

마무리

Model Assets에 등록된 모델들을 하나의 model service로 생성하여, rest app에서 호출하는 방법을 알아보았습니다.