閃 き

閃き- blog

きらびやかに、美しく、痛烈に．

Keras/Tensorflow : CIFAR-10のVGG-likeなアーキテクチャを作った.

Python Tensorflow 機械学習 Keras

VGG

1. 動作環境

OS: Ubuntu 16.04

Package             Version  
------------------- -------
python              3.5.0
tensorboard         1.9.0    
tensorflow          1.9.0    
h5py                2.8.0    
Keras               2.2.2    
Keras-Applications  1.0.4    
Keras-Preprocessing 1.0.2

2. プログラム

import os
import numpy as np
import keras.backend.tensorflow_backend as KTF
import tensorflow as tf
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation, Flatten
from keras.layers.normalization import BatchNormalization
from keras.layers import Conv2D, MaxPooling2D
from keras.optimizers import SGD, Adam
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import EarlyStopping, ModelCheckpoint, TensorBoard, CSVLogger
from datetime import datetime

# Set Meta Parameters & Information
img_row = 32.0
img_col = 32.0
img_ch  = 255.0

batch_size = 128
nb_classes = 10
nb_epoch   = 200
nb_data    = 32*32

log_dir        = '../train_log/vgg-like_log'
dataset_dir    = '../../CIFAR-10/datasets'
model_name     = 'vgg-like__' + datetime.now().strftime("%Y-%m-%d-%H-%M-%S")
model_cp_path  = os.path.join(log_dir, (model_name + '_checkpoint.h5'))
model_csv_path = os.path.join(log_dir, (model_name + '_csv.csv'))

# Load CIFAR-10 Dataset
argmax_ch  = 255.0
(X_train, y_train), (X_test, y_test) = cifar10.load_data()

# convert pixel value range 0.0-255.0 to 0.0-1.0
X_train = X_train.astype('float32') / img_ch
X_test  = X_test.astype('float32')  / img_ch

# convert class label (0-9) to one-hot encoding format
y_train = np_utils.to_categorical(y_train, nb_classes)
y_test  = np_utils.to_categorical(y_test, nb_classes)

# save datasets as "np.ndarray" format files
np.save('X_train', X_train)
np.save('y_train', y_train)
np.save('X_test' , X_test)
np.save('y_test' , y_test)

# Data Augumatation
datagen = ImageDataGenerator(
        featurewise_center=True,
        featurewise_std_normalization=True,
        rotation_range=20,
        width_shift_range=0.2,
        height_shift_range=0.2,
        horizontal_flip=True
        vertical_flip=False)
datagen.fit(X_train)



old_session = KTF.get_session()

with tf.Graph().as_default():
    session = tf.Session('')
    KTF.set_session(session)
    KTF.set_learning_phase(1)

    # build model
    model = Sequential()
    with tf.name_scope('inference') as scope:
        model.add(Conv2D(64, (3, 3), padding='same', input_shape=X_train.shape[1:]))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(Conv2D(64, (3, 3), padding='same'))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(MaxPooling2D(pool_size=(2, 2)))
        model.add(Dropout(0.25))

        model.add(Conv2D(128, (3, 3), padding='same'))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(Conv2D(128, (3, 3), padding='same'))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(MaxPooling2D(pool_size=(2, 2)))
        model.add(Dropout(0.25))

        model.add(Conv2D(256, (3, 3), padding='same'))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(Conv2D(256, (3, 3), padding='same'))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(Conv2D(256, (3, 3), padding='same'))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(Conv2D(256, (3, 3), padding='same'))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(MaxPooling2D(pool_size=(2, 2)))
        model.add(Dropout(0.25))

        model.add(Flatten())
        model.add(Dense(1024))
        model.add(Activation('relu'))
        model.add(Dropout(0.5))
        model.add(Dense(1024))
        model.add(Activation('relu'))
        model.add(Dropout(0.5))
        model.add(Dense(nb_classes))
        model.add(Activation('softmax'))
    model.summary()

    # set callbacks
    cp_cb  = ModelCheckpoint(model_cp_path, monitor='val_loss', save_best_only=True)
    tb_cb  = TensorBoard(log_dir=log_dir, histogram_freq=1)
    csv_cb = CSVLogger(model_csv_path) 
    callbacks = [cp_cb, tb_cb, csv_cb]

    # compile model
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    history = model.fit_generator(
        datagen.flow(X_train, y_train, batch_size=batch_size),
        steps_per_epoch=len(X_train) / batch_size,
        epochs=nb_epoch,
        verbose=1,
        callbacks=callbacks,
        validation_data=(X_test, y_test))
    
    # validation
    score = model.evaluate(X_test, y_test, verbose=0)
    print('val score    : ', score[0])
    print('val accuracy : ', score[1])


# save model "INSTANCE"
f1_name = model_name + '_instance'
f1_path = os.path.join(log_dir, f1_name) + '.h5'
model.save(f1_path)

# save model "WEIGHTs"
f2_name = model_name + '_weights'
f2_path = os.path.join(log_dir, f2_name) + '.h5'
model.save_weights(f2_path)

# save model "ARCHITECHTURE"
f3_name = model_name + '_architechture'
f3_path = os.path.join(log_dir, f3_name) + '.json'
json_string = model.to_json()
with open(f3_path, 'w') as f:
    f.write(json_string)


# end of session
KTF.set_session(old_session)

3.アーキテクチャ

Kerasのkeras.models.Model()クラスもつ属性(attribute)である"summary()"を使う.
プログラム上のmodel.summary()で、標準出力にモデルの構造(architechture)の要約情報が表示される.

Modelクラス (functional API) - Keras Documentation

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 32, 32, 64)        1792      
_________________________________________________________________
batch_normalization_1 (Batch (None, 32, 32, 64)        256       
_________________________________________________________________
activation_1 (Activation)    (None, 32, 32, 64)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 32, 32, 64)        36928     
_________________________________________________________________
batch_normalization_2 (Batch (None, 32, 32, 64)        256       
_________________________________________________________________
activation_2 (Activation)    (None, 32, 32, 64)        0         
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 16, 16, 64)        0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 16, 16, 64)        0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 16, 16, 128)       73856     
_________________________________________________________________
batch_normalization_3 (Batch (None, 16, 16, 128)       512       
_________________________________________________________________
activation_3 (Activation)    (None, 16, 16, 128)       0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 16, 16, 128)       147584    
_________________________________________________________________
batch_normalization_4 (Batch (None, 16, 16, 128)       512       
_________________________________________________________________
activation_4 (Activation)    (None, 16, 16, 128)       0         
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 8, 8, 128)         0         
_________________________________________________________________
dropout_2 (Dropout)          (None, 8, 8, 128)         0         
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 8, 8, 256)         295168    
_________________________________________________________________
batch_normalization_5 (Batch (None, 8, 8, 256)         1024      
_________________________________________________________________
activation_5 (Activation)    (None, 8, 8, 256)         0         
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 8, 8, 256)         590080    
_________________________________________________________________
batch_normalization_6 (Batch (None, 8, 8, 256)         1024      
_________________________________________________________________
activation_6 (Activation)    (None, 8, 8, 256)         0         
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 8, 8, 256)         590080    
_________________________________________________________________
batch_normalization_7 (Batch (None, 8, 8, 256)         1024      
_________________________________________________________________
activation_7 (Activation)    (None, 8, 8, 256)         0         
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 8, 8, 256)         590080    
_________________________________________________________________
batch_normalization_8 (Batch (None, 8, 8, 256)         1024      
_________________________________________________________________
activation_8 (Activation)    (None, 8, 8, 256)         0         
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 4, 4, 256)         0         
_________________________________________________________________
dropout_3 (Dropout)          (None, 4, 4, 256)         0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 4096)              0         
_________________________________________________________________
dense_1 (Dense)              (None, 1024)              4195328   
_________________________________________________________________
activation_9 (Activation)    (None, 1024)              0         
_________________________________________________________________
dropout_4 (Dropout)          (None, 1024)              0         
_________________________________________________________________
dense_2 (Dense)              (None, 1024)              1049600   
_________________________________________________________________
activation_10 (Activation)   (None, 1024)              0         
_________________________________________________________________
dropout_5 (Dropout)          (None, 1024)              0         
_________________________________________________________________
dense_3 (Dense)              (None, 10)                10250     
_________________________________________________________________
activation_11 (Activation)   (None, 10)                0         
=================================================================
Total params: 7,586,378
Trainable params: 7,583,562
Non-trainable params: 2,816
_________________________________________________________________


4.学習結果

validation accuracy は 90% くらい

f:id:yumaloop:20180822170332p:plain

f:id:yumaloop:20180822170409p:plain

Optimal Brain Damage : 局所二次近似と極値判定（凸関数, 勾配, ヘッセ行列）

数学 機械学習

http://3.bp.blogspot.com/-BN1iZIyQafs/T9RhL7lhcVI/AAAAAAAAAB0/AQV4tRb9ib0/s1600/x2my2.png


このポストでは，ニューラルネットワークモデルにおける「パラメータを訓練(学習)させるためのアルゴリズム」に注目したいと思います．ニューラルネットワークモデルについては，

「特徴量（=パラメータ）を人間が与えなくとも，機械(=コンピュータ)が勝手にその "重要度" を評価し，学習する」

という内容で，その新奇性を紹介されることが多いですが．この主張の要点は，

  • 各重み (params) の性能への"寄与度"を機械 (model) が自動的に評価すること
  • 各重み (params) の性能への"寄与度"を機械 (model) が評価した後に，その情報を機械 (model) に反映させること

の２つに集約できます．しかし，前者に関しては，適応信号処理や情報検索，機械学習などで既に多くの一般的な研究成果が与えられているため，ニューラルネットワーク特有の性質とは言えません．その意味で，個人的には，後者：「各重み (params) の性能への"寄与度"を機械 (model) が評価した後に，その情報を機械(model) に自動的に反映させること」こそがニューラルネットワークモデルのクリティカルな性質であると言って良いでしょう．そしてこれは，具体的に「誤差逆伝播法」というアイデアと，これを解くための「最急降下法」という最適化アルゴリズムに基づきます．

以上を踏まえ，このポストでは，ニューラルネットワークモデルの核である「誤差逆伝播法」「最急降下法」に注目した上で，その原理となる「局所二次近似」や「凸最適化」についてまとめてみたいと思います．


1. 背景

https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Monkey_saddle_surface.svg/300px-Monkey_saddle_surface.svg.png

Optimal Brain Damage (: OBD)とは，ニューラルネットワークを用いた目的変数の予測問題において、パラメータ数を縮減するための方法であり，局所二次近似 (Approximation of Second-order Derivatives) とは，OBDを実装する際に必要となる計算アルゴリズムのことです。具体的には，目的関数(=誤差関数, 損失関数)を２次の項までのテイラー展開によって近似することで，あるパラメータの組み合わせに対する「勾配」を求めます．

なお．OBDは、Yann Le Can らによって書かれた初期のニューラルネットワーク研究における有名な論文：Optimal Brain Damage [Yan, et al. 1990]によって提案されました．また，OBDに関連する概念・用語としては，以下のようなものが挙げられます．

  • OBD: Optimal Brain Damage
  • OBS: Optimal Brain Surgeon
  • OCD: Optimal Cell Damage


ここからは、局所二次近似について考えてみましょう．


2. 準備：凸関数とテイラー展開

まず，凸関数(Convex function)，勾配(Gradient)，ヘッセ行列(Hessian)の定義を与えます．

2. 1. 凸関数

n個のスカラー変量 \{x_n\} からなる多変数関数： f(x_1, x_2, ... x_n)~=f({\bf x}) を考える．

定義域上の任意の2点 {\bf x_a}, ~{\bf x_b} (n次元ベクトル)において，次の不等式を満たすとき， fを「凸関数」(: Convex function)という．

\begin{equation} {\lambda}{\cdot}f({\bf x_a})+(1-{\lambda}){\cdot}f({\bf x_b})~~{\geq}~~f({\lambda}{\cdot}{\bf x_a}+(1-{\lambda}){\cdot}{\bf x_b}) \end{equation}


なお，等号成立が {\bf x_a}~=~{\bf x_b}の場合に限るとき， f を「狭義の凸関数」という．


2. 2. 勾配 - Gradient

n個のスカラー変量 \{x_n\} からなる多変数関数： f(x_1, x_2, ... x_n)~=f({\bf x}) を考える．

f(x_1, x_2, ... x_n) が，任意の変数 x_1, x_2, ... x_n について1階偏導関数が定義されているとき， fに対する1階微分の情報を包括的に含む以下のn次元ベクトル \bf\nabla が定められる．これを「 fの勾配(gradient, grad f)」という．

{\bf\nabla}f({\bf x})=\left( \begin{array}{cccc} \frac{\partial f({\bf x})}{\partial x_1} \\ \frac{\partial f({\bf x})}{\partial x_2} \\ \vdots \\ \frac{\partial f({\bf x})}{\partial x_n} \end{array} \right)


{\bf\nabla}f({\bf x})={\large\frac{\partial f}{\partial x_1}}\mathbf{e_1}+{\large\frac{\partial f}{\partial x_2}}\mathbf{e_2}+\ldots+{\large\frac{\partial f}{\partial x_n}}\mathbf{e_n}



2. 3. ヘッセ行列 - Hessian Matrix

n個のスカラー変量 \{x_n\} からなる多変数関数： f(x_1, x_2, ... x_n)~=f({\bf x}) を考える．

f(x_1, x_2, ... x_n) が，任意の変数 x_1, x_2, ... x_n について2階偏導関数が定義されているとき， fに対する2階微分の情報を包括的に含む以下のn×n正方行列: {\bf H}~={\nabla^2}f({\bf x}) が定められる．これをヘッセ行列という．

{\bf H}={{\bf\nabla}^2}f({\bf x})=\left( \begin{array}{cccc} {\frac{{\partial}^2 f({\bf x})}{{\partial x_1}{\partial x_1}}} & {\frac{{\partial}^2 f({\bf x})}{{\partial x_1}{\partial x_2}}} & \ldots & {\frac{{\partial}^2 f({\bf x})}{{\partial x_1}{\partial x_n}}} \\ {\frac{{\partial}^2 f({\bf x})}{{\partial x_2}{\partial x_1}}} & {\frac{{\partial}^2 f({\bf x})}{{\partial x_2}{\partial x_2}}} & \ldots & {\frac{{\partial}^2 f({\bf x})}{{\partial x_2}{\partial x_n}}} \\ \vdots & \vdots & \ddots & \vdots \\ {\frac{{\partial}^2 f({\bf x})}{{\partial x_n}{\partial x_1}}} & {\frac{{\partial}^2 f({\bf x})}{{\partial x_n}{\partial x_2}}} & \ldots & {\frac{{\partial}^2 f({\bf x})}{{\partial x_n}{\partial x_n}}} \end{array} \right)

ただし，ヘッセ行列 {\bf H}の各要素は次のように定義される．

h_{ij}~=~\frac{{\partial}^2 f}{{\partial x_i}{\partial x_j}}



2. 4. テイラー展開による2次の近似

n個のスカラー変量 \{x_n\} からなる多変数関数： f(x_1, x_2, ... x_n)~=f({\bf x}) を考える．


テイラー展開より， f を任意の点 {\bf \hat{x}~(=a)} によって近似すると，次式を得る．

f({\bf x})= f({\bf a})+\sum_{n=1}^{\infty}{\large\frac{f^{(n)}({\bf a})}{n!}}({\bf x}-{\bf a})^n

f({\bf x}) = f({\bf a})+{\large\frac{f^{(1)}({\bf a})}{1!}}({\bf x}-{\bf a})+{\large\frac{f^{(2)}({\bf a})}{2!}}({\bf x}-{\bf a})^2+{\large\frac{f^{(3)}({\bf a})}{3!}}({\bf x}-{\bf a})^3+~\cdots



ここで、 f(x_1, x_2, ... x_n)~=f({\bf x})を、2次までのテイラー展開によって近似すると、

f({\bf x})~{\fallingdotseq}~f({\bf a})+{\large\frac{f^{(1)}({\bf a})}{1!}}({\bf x}-{\bf a})+{\large\frac{f^{(2)}({\bf a})}{2!}}({\bf x}-{\bf a})^2

となり、既知の値 {\bf \hat{x}~(=a)}によって、 f を推定することができる。



3. 局所二次近似

さて，ここまでの議論で共通して登場してきた

「n個のスカラー変量 \{x_n\} からなる多変数関数： f(x_1, x_2, ... x_n)~=f({\bf x})」

は，予測問題の文脈では

「n次元のベクトル {\bf w} に関する誤差関数(=目的関数)： E({(w_1, w_2, ... w_n)}^{\mathrm{T}})~=E({\bf w}) 」

とみなすことができます.


誤差関数 E({\bf x}) に対するパラメータベクトル {\bf w} に，既知の値 \hat{w} を与えて，2次のテイラー展開によって近似してみましょう．

E({\bf w})~{\fallingdotseq}~E({\bf \hat{w}})+{\large\frac{E^{(1)}({\bf \hat{w}})}{1!}}({\bf w}-{\bf \hat{w}})+{\large\frac{E^{(2)}({\bf \hat{w}})}{2!}}({\bf w}-{\bf \hat{w}})^2

  • 誤差関数を最小にするようなパラメータベクトル {\bf w} を求めたい.
  • 誤差関数はパラメータベクトル {\bf w} に対して凸関数となる．

誤差関数の

coming soon ..



4. 極値判定

x において正定値対称行列であるとき、f は x において極小である。
x において負定値対称行列であるとき、f は x において極大である。
x において正負両方の固有値を持つとき、x は f の鞍点である（これは x が退化する場合にも正しい）。

coming soon ..

参考

Javaで文字列として使える物理フォント一覧を表示させる方法

Java 備忘 計算機環境

java.awt.GraphicsEnvironmentクラスにある
getLocalGraphicsEnvironment().getAllFonts();メソッドを使う。

GraphicsEnvironment (Java Platform SE 8)

・コードサンプル

import java.awt.Font;
import java.awt.GraphicsEnvironment;

public class Main {
	public static void main(String[] args) throws Exception {
	    Font[] fonts  = GraphicsEnvironment.getLocalGraphicsEnvironment().getAllFonts();
	    for (int i = 0; i < fonts.length; i++) {
	      System.out.print(fonts[i].getFontName() + ", ");
	      System.out.print(fonts[i].getFamily() + ", ");
	      System.out.print(fonts[i].getName());
	      System.out.println();
	    }
	}	
}


・実行結果

Serif, Serif, Serif, 
SansSerif, SansSerif, SansSerif, 
Monospaced, Monospaced, Monospaced, 
Dialog, Dialog, Dialog, 
DialogInput, DialogInput, DialogInput, 
.SFNSText, .SF NS Text, .SFNSText, 
.SFNSText-Bold, .SF NS Text, .SFNSText-Bold, 
AlBayan, Al Bayan, AlBayan, 
AlBayan-Bold, Al Bayan, AlBayan-Bold, 
AlNile, Al Nile, AlNile, 
AlNile-Bold, Al Nile, AlNile-Bold, 
AlTarikh, Al Tarikh, AlTarikh, 
AmericanTypewriter, American Typewriter, AmericanTypewriter, 
AmericanTypewriter-Bold, American Typewriter, AmericanTypewriter-Bold, 
AmericanTypewriter-Condensed, American Typewriter, AmericanTypewriter-Condensed, 
AmericanTypewriter-CondensedBold, American Typewriter, AmericanTypewriter-CondensedBold, 
AmericanTypewriter-CondensedLight, American Typewriter, AmericanTypewriter-CondensedLight, 
AmericanTypewriter-Light, American Typewriter, AmericanTypewriter-Light, 
AmericanTypewriter-Semibold, American Typewriter, AmericanTypewriter-Semibold, 
AndaleMono, Andale Mono, AndaleMono, 
Apple-Chancery, Apple Chancery, Apple-Chancery, 
AppleBraille, Apple Braille, AppleBraille, 
AppleBraille-Outline6Dot, Apple Braille, AppleBraille-Outline6Dot, 
AppleBraille-Outline8Dot, Apple Braille, AppleBraille-Outline8Dot, 
AppleBraille-Pinpoint6Dot, Apple Braille, AppleBraille-Pinpoint6Dot, 
AppleBraille-Pinpoint8Dot, Apple Braille, AppleBraille-Pinpoint8Dot, 
AppleColorEmoji, Apple Color Emoji, AppleColorEmoji, 
AppleGothic, AppleGothic, AppleGothic, 
AppleMyungjo, AppleMyungjo, AppleMyungjo, 
AppleSDGothicNeo-Bold, Apple SD Gothic Neo, AppleSDGothicNeo-Bold, 
AppleSDGothicNeo-ExtraBold, Apple SD Gothic Neo, AppleSDGothicNeo-ExtraBold, 
AppleSDGothicNeo-Heavy, Apple SD Gothic Neo, AppleSDGothicNeo-Heavy, 
AppleSDGothicNeo-Light, Apple SD Gothic Neo, AppleSDGothicNeo-Light, 
AppleSDGothicNeo-Medium, Apple SD Gothic Neo, AppleSDGothicNeo-Medium, 
AppleSDGothicNeo-Regular, Apple SD Gothic Neo, AppleSDGothicNeo-Regular, 
AppleSDGothicNeo-SemiBold, Apple SD Gothic Neo, AppleSDGothicNeo-SemiBold, 
AppleSDGothicNeo-Thin, Apple SD Gothic Neo, AppleSDGothicNeo-Thin, 
AppleSDGothicNeo-UltraLight, Apple SD Gothic Neo, AppleSDGothicNeo-UltraLight, 
AppleSymbols, Apple Symbols, AppleSymbols, 
Arial-Black, Arial Black, Arial-Black, 
Arial-BoldItalicMT, Arial, Arial-BoldItalicMT, 
Arial-BoldMT, Arial, Arial-BoldMT, 
Arial-ItalicMT, Arial, Arial-ItalicMT, 
ArialHebrew, Arial Hebrew, ArialHebrew, 
ArialHebrew-Bold, Arial Hebrew, ArialHebrew-Bold, 
ArialHebrew-Light, Arial Hebrew, ArialHebrew-Light, 
ArialHebrewScholar, Arial Hebrew Scholar, ArialHebrewScholar, 
ArialHebrewScholar-Bold, Arial Hebrew Scholar, ArialHebrewScholar-Bold, 
ArialHebrewScholar-Light, Arial Hebrew Scholar, ArialHebrewScholar-Light, 
ArialMT, Arial, ArialMT, 
ArialNarrow, Arial Narrow, ArialNarrow, 
ArialNarrow-Bold, Arial Narrow, ArialNarrow-Bold, 
ArialNarrow-BoldItalic, Arial Narrow, ArialNarrow-BoldItalic, 
ArialNarrow-Italic, Arial Narrow, ArialNarrow-Italic, 
ArialRoundedMTBold, Arial Rounded MT Bold, ArialRoundedMTBold, 
ArialUnicodeMS, Arial Unicode MS, ArialUnicodeMS, 
Athelas-Bold, Athelas, Athelas-Bold, 
Athelas-BoldItalic, Athelas, Athelas-BoldItalic, 
Athelas-Italic, Athelas, Athelas-Italic, 
Athelas-Regular, Athelas, Athelas-Regular, 
Avenir-Black, Avenir, Avenir-Black, 
Avenir-BlackOblique, Avenir, Avenir-BlackOblique, 
Avenir-Book, Avenir, Avenir-Book, 
Avenir-BookOblique, Avenir, Avenir-BookOblique, 
Avenir-Heavy, Avenir, Avenir-Heavy, 
Avenir-HeavyOblique, Avenir, Avenir-HeavyOblique, 
Avenir-Light, Avenir, Avenir-Light, 
Avenir-LightOblique, Avenir, Avenir-LightOblique, 
Avenir-Medium, Avenir, Avenir-Medium, 
Avenir-MediumOblique, Avenir, Avenir-MediumOblique, 
Avenir-Oblique, Avenir, Avenir-Oblique, 
Avenir-Roman, Avenir, Avenir-Roman, 
AvenirNext-Bold, Avenir Next, AvenirNext-Bold, 
AvenirNext-BoldItalic, Avenir Next, AvenirNext-BoldItalic, 
AvenirNext-DemiBold, Avenir Next, AvenirNext-DemiBold, 
AvenirNext-DemiBoldItalic, Avenir Next, AvenirNext-DemiBoldItalic, 
AvenirNext-Heavy, Avenir Next, AvenirNext-Heavy, 
AvenirNext-HeavyItalic, Avenir Next, AvenirNext-HeavyItalic, 
AvenirNext-Italic, Avenir Next, AvenirNext-Italic, 
AvenirNext-Medium, Avenir Next, AvenirNext-Medium, 
AvenirNext-MediumItalic, Avenir Next, AvenirNext-MediumItalic, 
AvenirNext-Regular, Avenir Next, AvenirNext-Regular, 
AvenirNext-UltraLight, Avenir Next, AvenirNext-UltraLight, 
AvenirNext-UltraLightItalic, Avenir Next, AvenirNext-UltraLightItalic, 
AvenirNextCondensed-Bold, Avenir Next Condensed, AvenirNextCondensed-Bold, 
AvenirNextCondensed-BoldItalic, Avenir Next Condensed, AvenirNextCondensed-BoldItalic, 
AvenirNextCondensed-DemiBold, Avenir Next Condensed, AvenirNextCondensed-DemiBold, 
AvenirNextCondensed-DemiBoldItalic, Avenir Next Condensed, AvenirNextCondensed-DemiBoldItalic, 
AvenirNextCondensed-Heavy, Avenir Next Condensed, AvenirNextCondensed-Heavy, 
AvenirNextCondensed-HeavyItalic, Avenir Next Condensed, AvenirNextCondensed-HeavyItalic, 
AvenirNextCondensed-Italic, Avenir Next Condensed, AvenirNextCondensed-Italic, 
AvenirNextCondensed-Medium, Avenir Next Condensed, AvenirNextCondensed-Medium, 
AvenirNextCondensed-MediumItalic, Avenir Next Condensed, AvenirNextCondensed-MediumItalic, 
AvenirNextCondensed-Regular, Avenir Next Condensed, AvenirNextCondensed-Regular, 
AvenirNextCondensed-UltraLight, Avenir Next Condensed, AvenirNextCondensed-UltraLight, 
AvenirNextCondensed-UltraLightItalic, Avenir Next Condensed, AvenirNextCondensed-UltraLightItalic, 
Ayuthaya, Ayuthaya, Ayuthaya, 
Baghdad, Baghdad, Baghdad, 
BanglaMN, Bangla MN, BanglaMN, 
BanglaMN-Bold, Bangla MN, BanglaMN-Bold, 
BanglaSangamMN, Bangla Sangam MN, BanglaSangamMN, 
BanglaSangamMN-Bold, Bangla Sangam MN, BanglaSangamMN-Bold, 
Baskerville, Baskerville, Baskerville, 
Baskerville-Bold, Baskerville, Baskerville-Bold, 
Baskerville-BoldItalic, Baskerville, Baskerville-BoldItalic, 
Baskerville-Italic, Baskerville, Baskerville-Italic, 
Baskerville-SemiBold, Baskerville, Baskerville-SemiBold, 
Baskerville-SemiBoldItalic, Baskerville, Baskerville-SemiBoldItalic, 
Beirut, Beirut, Beirut, 
BigCaslon-Medium, Big Caslon, BigCaslon-Medium, 
BodoniOrnamentsITCTT, Bodoni Ornaments, BodoniOrnamentsITCTT, 
BodoniSvtyTwoITCTT-Bold, Bodoni 72, BodoniSvtyTwoITCTT-Bold, 
BodoniSvtyTwoITCTT-Book, Bodoni 72, BodoniSvtyTwoITCTT-Book, 
BodoniSvtyTwoITCTT-BookIta, Bodoni 72, BodoniSvtyTwoITCTT-BookIta, 
BodoniSvtyTwoOSITCTT-Bold, Bodoni 72 Oldstyle, BodoniSvtyTwoOSITCTT-Bold, 
BodoniSvtyTwoOSITCTT-Book, Bodoni 72 Oldstyle, BodoniSvtyTwoOSITCTT-Book, 
BodoniSvtyTwoOSITCTT-BookIt, Bodoni 72 Oldstyle, BodoniSvtyTwoOSITCTT-BookIt, 
BodoniSvtyTwoSCITCTT-Book, Bodoni 72 Smallcaps, BodoniSvtyTwoSCITCTT-Book, 
BradleyHandITCTT-Bold, Bradley Hand, BradleyHandITCTT-Bold, 
BrushScriptMT, Brush Script MT, BrushScriptMT, 
Chalkboard, Chalkboard, Chalkboard, 
Chalkboard-Bold, Chalkboard, Chalkboard-Bold, 
ChalkboardSE-Bold, Chalkboard SE, ChalkboardSE-Bold, 
ChalkboardSE-Light, Chalkboard SE, ChalkboardSE-Light, 
ChalkboardSE-Regular, Chalkboard SE, ChalkboardSE-Regular, 
Chalkduster, Chalkduster, Chalkduster, 
Charter-Black, Charter, Charter-Black, 
Charter-BlackItalic, Charter, Charter-BlackItalic, 
Charter-Bold, Charter, Charter-Bold, 
Charter-BoldItalic, Charter, Charter-BoldItalic, 
Charter-Italic, Charter, Charter-Italic, 
Charter-Roman, Charter, Charter-Roman, 
Cochin, Cochin, Cochin, 
Cochin-Bold, Cochin, Cochin-Bold, 
Cochin-BoldItalic, Cochin, Cochin-BoldItalic, 
Cochin-Italic, Cochin, Cochin-Italic, 
ComicSansMS, Comic Sans MS, ComicSansMS, 
ComicSansMS-Bold, Comic Sans MS, ComicSansMS-Bold, 
Copperplate, Copperplate, Copperplate, 
Copperplate-Bold, Copperplate, Copperplate-Bold, 
Copperplate-Light, Copperplate, Copperplate-Light, 
CorsivaHebrew, Corsiva Hebrew, CorsivaHebrew, 
CorsivaHebrew-Bold, Corsiva Hebrew, CorsivaHebrew-Bold, 
Courier, Courier, Courier, 
Courier-Bold, Courier, Courier-Bold, 
Courier-BoldOblique, Courier, Courier-BoldOblique, 
Courier-Oblique, Courier, Courier-Oblique, 
CourierNewPS-BoldItalicMT, Courier New, CourierNewPS-BoldItalicMT, 
CourierNewPS-BoldMT, Courier New, CourierNewPS-BoldMT, 
CourierNewPS-ItalicMT, Courier New, CourierNewPS-ItalicMT, 
CourierNewPSMT, Courier New, CourierNewPSMT, 
DFKaiShu-SB-Estd-BF, BiauKai, DFKaiShu-SB-Estd-BF, 
DFWaWaSC-W5, Wawati SC, DFWaWaSC-W5, 
DFWaWaTC-W5, Wawati TC, DFWaWaTC-W5, 
DINAlternate-Bold, DIN Alternate, DINAlternate-Bold, 
DINCondensed-Bold, DIN Condensed, DINCondensed-Bold, 
Damascus, Damascus, Damascus, 
DamascusBold, Damascus, DamascusBold, 
DamascusLight, Damascus, DamascusLight, 
DamascusMedium, Damascus, DamascusMedium, 
DamascusSemiBold, Damascus, DamascusSemiBold, 
DecoTypeNaskh, DecoType Naskh, DecoTypeNaskh, 
DevanagariMT, Devanagari MT, DevanagariMT, 
DevanagariMT-Bold, Devanagari MT, DevanagariMT-Bold, 
DevanagariSangamMN, Devanagari Sangam MN, DevanagariSangamMN, 
DevanagariSangamMN-Bold, Devanagari Sangam MN, DevanagariSangamMN-Bold, 
Didot, Didot, Didot, 
Didot-Bold, Didot, Didot-Bold, 
Didot-Italic, Didot, Didot-Italic, 
DiwanKufi, Diwan Kufi, DiwanKufi, 
DiwanMishafi, Mishafi, DiwanMishafi, 
DiwanMishafiGold, Mishafi Gold, DiwanMishafiGold, 
DiwanThuluth, Diwan Thuluth, DiwanThuluth, 
EuphemiaUCAS, Euphemia UCAS, EuphemiaUCAS, 
EuphemiaUCAS-Bold, Euphemia UCAS, EuphemiaUCAS-Bold, 
EuphemiaUCAS-Italic, Euphemia UCAS, EuphemiaUCAS-Italic, 
FZLTTHB--B51-0, Lantinghei TC, FZLTTHB--B51-0, 
FZLTTHK--GBK1-0, Lantinghei SC, FZLTTHK--GBK1-0, 
FZLTXHB--B51-0, Lantinghei TC, FZLTXHB--B51-0, 
FZLTXHK--GBK1-0, Lantinghei SC, FZLTXHK--GBK1-0, 
FZLTZHB--B51-0, Lantinghei TC, FZLTZHB--B51-0, 
FZLTZHK--GBK1-0, Lantinghei SC, FZLTZHK--GBK1-0, 
Farah, Farah, Farah, 
Farisi, Farisi, Farisi, 
Futura-Bold, Futura, Futura-Bold, 
Futura-CondensedExtraBold, Futura, Futura-CondensedExtraBold, 
Futura-CondensedMedium, Futura, Futura-CondensedMedium, 
Futura-Medium, Futura, Futura-Medium, 
Futura-MediumItalic, Futura, Futura-MediumItalic, 
GB18030Bitmap, GB18030 Bitmap, GB18030Bitmap, 
GeezaPro, Geeza Pro, GeezaPro, 
GeezaPro-Bold, Geeza Pro, GeezaPro-Bold, 
Geneva, Geneva, Geneva, 
Georgia, Georgia, Georgia, 
Georgia-Bold, Georgia, Georgia-Bold, 
Georgia-BoldItalic, Georgia, Georgia-BoldItalic, 
Georgia-Italic, Georgia, Georgia-Italic, 
GillSans, Gill Sans, GillSans, 
GillSans-Bold, Gill Sans, GillSans-Bold, 
GillSans-BoldItalic, Gill Sans, GillSans-BoldItalic, 
GillSans-Italic, Gill Sans, GillSans-Italic, 
GillSans-Light, Gill Sans, GillSans-Light, 
GillSans-LightItalic, Gill Sans, GillSans-LightItalic, 
GillSans-SemiBold, Gill Sans, GillSans-SemiBold, 
GillSans-SemiBoldItalic, Gill Sans, GillSans-SemiBoldItalic, 
GillSans-UltraBold, Gill Sans, GillSans-UltraBold, 
GujaratiMT, Gujarati MT, GujaratiMT, 
GujaratiMT-Bold, Gujarati MT, GujaratiMT-Bold, 
GujaratiSangamMN, Gujarati Sangam MN, GujaratiSangamMN, 
GujaratiSangamMN-Bold, Gujarati Sangam MN, GujaratiSangamMN-Bold, 
GurmukhiMN, Gurmukhi MN, GurmukhiMN, 
GurmukhiMN-Bold, Gurmukhi MN, GurmukhiMN-Bold, 
GurmukhiSangamMN, Gurmukhi Sangam MN, GurmukhiSangamMN, 
GurmukhiSangamMN-Bold, Gurmukhi Sangam MN, GurmukhiSangamMN-Bold, 
HannotateSC-W5, Hannotate SC, HannotateSC-W5, 
HannotateSC-W7, Hannotate SC, HannotateSC-W7, 
HannotateTC-W5, Hannotate TC, HannotateTC-W5, 
HannotateTC-W7, Hannotate TC, HannotateTC-W7, 
HanziPenSC-W3, HanziPen SC, HanziPenSC-W3, 
HanziPenSC-W5, HanziPen SC, HanziPenSC-W5, 
HanziPenTC-W3, HanziPen TC, HanziPenTC-W3, 
HanziPenTC-W5, HanziPen TC, HanziPenTC-W5, 
Helvetica, Helvetica, Helvetica, 
Helvetica-Bold, Helvetica, Helvetica-Bold, 
Helvetica-BoldOblique, Helvetica, Helvetica-BoldOblique, 
Helvetica-Light, Helvetica, Helvetica-Light, 
Helvetica-LightOblique, Helvetica, Helvetica-LightOblique, 
Helvetica-Oblique, Helvetica, Helvetica-Oblique, 
HelveticaNeue, Helvetica Neue, HelveticaNeue, 
HelveticaNeue-Bold, Helvetica Neue, HelveticaNeue-Bold, 
HelveticaNeue-BoldItalic, Helvetica Neue, HelveticaNeue-BoldItalic, 
HelveticaNeue-CondensedBlack, Helvetica Neue, HelveticaNeue-CondensedBlack, 
HelveticaNeue-CondensedBold, Helvetica Neue, HelveticaNeue-CondensedBold, 
HelveticaNeue-Italic, Helvetica Neue, HelveticaNeue-Italic, 
HelveticaNeue-Light, Helvetica Neue, HelveticaNeue-Light, 
HelveticaNeue-LightItalic, Helvetica Neue, HelveticaNeue-LightItalic, 
HelveticaNeue-Medium, Helvetica Neue, HelveticaNeue-Medium, 
HelveticaNeue-MediumItalic, Helvetica Neue, HelveticaNeue-MediumItalic, 
HelveticaNeue-Thin, Helvetica Neue, HelveticaNeue-Thin, 
HelveticaNeue-ThinItalic, Helvetica Neue, HelveticaNeue-ThinItalic, 
HelveticaNeue-UltraLight, Helvetica Neue, HelveticaNeue-UltraLight, 
HelveticaNeue-UltraLightItalic, Helvetica Neue, HelveticaNeue-UltraLightItalic, 
Herculanum, Herculanum, Herculanum, 
HiraKakuPro-W3, Hiragino Kaku Gothic Pro, HiraKakuPro-W3, 
HiraKakuPro-W6, Hiragino Kaku Gothic Pro, HiraKakuPro-W6, 
HiraKakuProN-W3, Hiragino Kaku Gothic ProN, HiraKakuProN-W3, 
HiraKakuProN-W6, Hiragino Kaku Gothic ProN, HiraKakuProN-W6, 
HiraKakuStd-W8, Hiragino Kaku Gothic Std, HiraKakuStd-W8, 
HiraKakuStdN-W8, Hiragino Kaku Gothic StdN, HiraKakuStdN-W8, 
HiraMaruPro-W4, Hiragino Maru Gothic Pro, HiraMaruPro-W4, 
HiraMaruProN-W4, Hiragino Maru Gothic ProN, HiraMaruProN-W4, 
HiraMinPro-W3, Hiragino Mincho Pro, HiraMinPro-W3, 
HiraMinPro-W6, Hiragino Mincho Pro, HiraMinPro-W6, 
HiraMinProN-W3, Hiragino Mincho ProN, HiraMinProN-W3, 
HiraMinProN-W6, Hiragino Mincho ProN, HiraMinProN-W6, 
HiraginoSans-W0, Hiragino Sans, HiraginoSans-W0, 
HiraginoSans-W1, Hiragino Sans, HiraginoSans-W1, 
HiraginoSans-W2, Hiragino Sans, HiraginoSans-W2, 
HiraginoSans-W3, Hiragino Sans, HiraginoSans-W3, 
HiraginoSans-W4, Hiragino Sans, HiraginoSans-W4, 
HiraginoSans-W5, Hiragino Sans, HiraginoSans-W5, 
HiraginoSans-W6, Hiragino Sans, HiraginoSans-W6, 
HiraginoSans-W7, Hiragino Sans, HiraginoSans-W7, 
HiraginoSans-W8, Hiragino Sans, HiraginoSans-W8, 
HiraginoSans-W9, Hiragino Sans, HiraginoSans-W9, 
HiraginoSansCNS-W3, Hiragino Sans CNS, HiraginoSansCNS-W3, 
HiraginoSansCNS-W6, Hiragino Sans CNS, HiraginoSansCNS-W6, 
HiraginoSansGB-W3, Hiragino Sans GB, HiraginoSansGB-W3, 
HiraginoSansGB-W6, Hiragino Sans GB, HiraginoSansGB-W6, 
HoeflerText-Black, Hoefler Text, HoeflerText-Black, 
HoeflerText-BlackItalic, Hoefler Text, HoeflerText-BlackItalic, 
HoeflerText-Italic, Hoefler Text, HoeflerText-Italic, 
HoeflerText-Ornaments, Hoefler Text, HoeflerText-Ornaments, 
HoeflerText-Regular, Hoefler Text, HoeflerText-Regular, 
ITFDevanagari-Bold, ITF Devanagari, ITFDevanagari-Bold, 
ITFDevanagari-Book, ITF Devanagari, ITFDevanagari-Book, 
ITFDevanagari-Demi, ITF Devanagari, ITFDevanagari-Demi, 
ITFDevanagari-Light, ITF Devanagari, ITFDevanagari-Light, 
ITFDevanagari-Medium, ITF Devanagari, ITFDevanagari-Medium, 
ITFDevanagariMarathi-Bold, ITF Devanagari Marathi, ITFDevanagariMarathi-Bold, 
ITFDevanagariMarathi-Book, ITF Devanagari Marathi, ITFDevanagariMarathi-Book, 
ITFDevanagariMarathi-Demi, ITF Devanagari Marathi, ITFDevanagariMarathi-Demi, 
ITFDevanagariMarathi-Light, ITF Devanagari Marathi, ITFDevanagariMarathi-Light, 
ITFDevanagariMarathi-Medium, ITF Devanagari Marathi, ITFDevanagariMarathi-Medium, 
Impact, Impact, Impact, 
InaiMathi, InaiMathi, InaiMathi, 
InaiMathi-Bold, InaiMathi, InaiMathi-Bold, 
IowanOldStyle-Black, Iowan Old Style, IowanOldStyle-Black, 
IowanOldStyle-BlackItalic, Iowan Old Style, IowanOldStyle-BlackItalic, 
IowanOldStyle-Bold, Iowan Old Style, IowanOldStyle-Bold, 
IowanOldStyle-BoldItalic, Iowan Old Style, IowanOldStyle-BoldItalic, 
IowanOldStyle-Italic, Iowan Old Style, IowanOldStyle-Italic, 
IowanOldStyle-Roman, Iowan Old Style, IowanOldStyle-Roman, 
IowanOldStyle-Titling, Iowan Old Style, IowanOldStyle-Titling, 
JCHEadA, HeadLineA, JCHEadA, 
JCfg, PilGi, JCfg, 
JCkg, GungSeo, JCkg, 
JCsmPC, PCMyungjo, JCsmPC, 
Kailasa, Kailasa, Kailasa, 
Kailasa-Bold, Kailasa, Kailasa-Bold, 
KannadaMN, Kannada MN, KannadaMN, 
KannadaMN-Bold, Kannada MN, KannadaMN-Bold, 
KannadaSangamMN, Kannada Sangam MN, KannadaSangamMN, 
KannadaSangamMN-Bold, Kannada Sangam MN, KannadaSangamMN-Bold, 
Kefa-Bold, Kefa, Kefa-Bold, 
Kefa-Regular, Kefa, Kefa-Regular, 
KhmerMN, Khmer MN, KhmerMN, 
KhmerMN-Bold, Khmer MN, KhmerMN-Bold, 
KhmerSangamMN, Khmer Sangam MN, KhmerSangamMN, 
Klee-Demibold, Klee, Klee-Demibold, 
Klee-Medium, Klee, Klee-Medium, 
KohinoorBangla-Bold, Kohinoor Bangla, KohinoorBangla-Bold, 
KohinoorBangla-Light, Kohinoor Bangla, KohinoorBangla-Light, 
KohinoorBangla-Medium, Kohinoor Bangla, KohinoorBangla-Medium, 
KohinoorBangla-Regular, Kohinoor Bangla, KohinoorBangla-Regular, 
KohinoorBangla-Semibold, Kohinoor Bangla, KohinoorBangla-Semibold, 
KohinoorDevanagari-Bold, Kohinoor Devanagari, KohinoorDevanagari-Bold, 
KohinoorDevanagari-Light, Kohinoor Devanagari, KohinoorDevanagari-Light, 
KohinoorDevanagari-Medium, Kohinoor Devanagari, KohinoorDevanagari-Medium, 
KohinoorDevanagari-Regular, Kohinoor Devanagari, KohinoorDevanagari-Regular, 
KohinoorDevanagari-Semibold, Kohinoor Devanagari, KohinoorDevanagari-Semibold, 
KohinoorTelugu-Bold, Kohinoor Telugu, KohinoorTelugu-Bold, 
KohinoorTelugu-Light, Kohinoor Telugu, KohinoorTelugu-Light, 
KohinoorTelugu-Medium, Kohinoor Telugu, KohinoorTelugu-Medium, 
KohinoorTelugu-Regular, Kohinoor Telugu, KohinoorTelugu-Regular, 
KohinoorTelugu-Semibold, Kohinoor Telugu, KohinoorTelugu-Semibold, 
Kokonor, Kokonor, Kokonor, 
Krungthep, Krungthep, Krungthep, 
KufiStandardGK, KufiStandardGK, KufiStandardGK, 
LaoMN, Lao MN, LaoMN, 
LaoMN-Bold, Lao MN, LaoMN-Bold, 
LaoSangamMN, Lao Sangam MN, LaoSangamMN, 
LiGothicMed, Apple LiGothic, LiGothicMed, 
LiHeiPro, LiHei Pro, LiHeiPro, 
LiSongPro, LiSong Pro, LiSongPro, 
LiSungLight, Apple LiSung, LiSungLight, 
Lucida Bright Demibold, Lucida Bright, Lucida Bright Demibold, 
Lucida Bright Demibold Italic, Lucida Bright, Lucida Bright Demibold Italic, 
Lucida Bright Italic, Lucida Bright, Lucida Bright Italic, 
Lucida Bright Regular, Lucida Bright, Lucida Bright Regular, 
Lucida Sans Demibold, Lucida Sans, Lucida Sans Demibold, 
Lucida Sans Regular, Lucida Sans, Lucida Sans Regular, 
Lucida Sans Typewriter Bold, Lucida Sans Typewriter, Lucida Sans Typewriter Bold, 
Lucida Sans Typewriter Regular, Lucida Sans Typewriter, Lucida Sans Typewriter Regular, 
LucidaBright, Lucida Bright, LucidaBright, 
LucidaBright-Demi, Lucida Bright, LucidaBright-Demi, 
LucidaBright-DemiItalic, Lucida Bright, LucidaBright-DemiItalic, 
LucidaBright-Italic, Lucida Bright, LucidaBright-Italic, 
LucidaGrande, Lucida Grande, LucidaGrande, 
LucidaGrande-Bold, Lucida Grande, LucidaGrande-Bold, 
LucidaSans, Lucida Sans, LucidaSans, 
LucidaSans-Demi, Lucida Sans, LucidaSans-Demi, 
LucidaSans-Typewriter, Lucida Sans Typewriter, LucidaSans-Typewriter, 
LucidaSans-TypewriterBold, Lucida Sans Typewriter, LucidaSans-TypewriterBold, 
Luminari-Regular, Luminari, Luminari-Regular, 
MLingWaiMedium-SC, LingWai SC, MLingWaiMedium-SC, 
MLingWaiMedium-TC, LingWai TC, MLingWaiMedium-TC, 
MalayalamMN, Malayalam MN, MalayalamMN, 
MalayalamMN-Bold, Malayalam MN, MalayalamMN-Bold, 
MalayalamSangamMN, Malayalam Sangam MN, MalayalamSangamMN, 
MalayalamSangamMN-Bold, Malayalam Sangam MN, MalayalamSangamMN-Bold, 
Marion-Bold, Marion, Marion-Bold, 
Marion-Italic, Marion, Marion-Italic, 
Marion-Regular, Marion, Marion-Regular, 
MarkerFelt-Thin, Marker Felt, MarkerFelt-Thin, 
MarkerFelt-Wide, Marker Felt, MarkerFelt-Wide, 
Menlo-Bold, Menlo, Menlo-Bold, 
Menlo-BoldItalic, Menlo, Menlo-BoldItalic, 
Menlo-Italic, Menlo, Menlo-Italic, 
Menlo-Regular, Menlo, Menlo-Regular, 
MicrosoftSansSerif, Microsoft Sans Serif, MicrosoftSansSerif, 
Monaco, Monaco, Monaco, 
MonotypeGurmukhi, Gurmukhi MT, MonotypeGurmukhi, 
Mshtakan, Mshtakan, Mshtakan, 
MshtakanBold, Mshtakan, MshtakanBold, 
MshtakanBoldOblique, Mshtakan, MshtakanBoldOblique, 
MshtakanOblique, Mshtakan, MshtakanOblique, 
Muna, Muna, Muna, 
MunaBlack, Muna, MunaBlack, 
MunaBold, Muna, MunaBold, 
MyanmarMN, Myanmar MN, MyanmarMN, 
MyanmarMN-Bold, Myanmar MN, MyanmarMN-Bold, 
MyanmarSangamMN, Myanmar Sangam MN, MyanmarSangamMN, 
MyanmarSangamMN-Bold, Myanmar Sangam MN, MyanmarSangamMN-Bold, 
Nadeem, Nadeem, Nadeem, 
NanumBrush, Nanum Brush Script, NanumBrush, 
NanumGothic, Nanum Gothic, NanumGothic, 
NanumGothicBold, Nanum Gothic, NanumGothicBold, 
NanumGothicExtraBold, Nanum Gothic, NanumGothicExtraBold, 
NanumMyeongjo, Nanum Myeongjo, NanumMyeongjo, 
NanumMyeongjoBold, Nanum Myeongjo, NanumMyeongjoBold, 
NanumMyeongjoExtraBold, Nanum Myeongjo, NanumMyeongjoExtraBold, 
NanumPen, Nanum Pen Script, NanumPen, 
NewPeninimMT, New Peninim MT, NewPeninimMT, 
NewPeninimMT-Bold, New Peninim MT, NewPeninimMT-Bold, 
NewPeninimMT-BoldInclined, New Peninim MT, NewPeninimMT-BoldInclined, 
NewPeninimMT-Inclined, New Peninim MT, NewPeninimMT-Inclined, 
Noteworthy-Bold, Noteworthy, Noteworthy-Bold, 
Noteworthy-Light, Noteworthy, Noteworthy-Light, 
NotoNastaliqUrdu, Noto Nastaliq Urdu, NotoNastaliqUrdu, 
Optima-Bold, Optima, Optima-Bold, 
Optima-BoldItalic, Optima, Optima-BoldItalic, 
Optima-ExtraBlack, Optima, Optima-ExtraBlack, 
Optima-Italic, Optima, Optima-Italic, 
Optima-Regular, Optima, Optima-Regular, 
OriyaMN, Oriya MN, OriyaMN, 
OriyaMN-Bold, Oriya MN, OriyaMN-Bold, 
OriyaSangamMN, Oriya Sangam MN, OriyaSangamMN, 
OriyaSangamMN-Bold, Oriya Sangam MN, OriyaSangamMN-Bold, 
Osaka, Osaka, Osaka, 
Osaka-Mono, Osaka, Osaka-Mono, 
PTMono-Bold, PT Mono, PTMono-Bold, 
PTMono-Regular, PT Mono, PTMono-Regular, 
PTSans-Bold, PT Sans, PTSans-Bold, 
PTSans-BoldItalic, PT Sans, PTSans-BoldItalic, 
PTSans-Caption, PT Sans Caption, PTSans-Caption, 
PTSans-CaptionBold, PT Sans Caption, PTSans-CaptionBold, 
PTSans-Italic, PT Sans, PTSans-Italic, 
PTSans-Narrow, PT Sans Narrow, PTSans-Narrow, 
PTSans-NarrowBold, PT Sans Narrow, PTSans-NarrowBold, 
PTSans-Regular, PT Sans, PTSans-Regular, 
PTSerif-Bold, PT Serif, PTSerif-Bold, 
PTSerif-BoldItalic, PT Serif, PTSerif-BoldItalic, 
PTSerif-Caption, PT Serif Caption, PTSerif-Caption, 
PTSerif-CaptionItalic, PT Serif Caption, PTSerif-CaptionItalic, 
PTSerif-Italic, PT Serif, PTSerif-Italic, 
PTSerif-Regular, PT Serif, PTSerif-Regular, 
Palatino-Bold, Palatino, Palatino-Bold, 
Palatino-BoldItalic, Palatino, Palatino-BoldItalic, 
Palatino-Italic, Palatino, Palatino-Italic, 
Palatino-Roman, Palatino, Palatino-Roman, 
Papyrus, Papyrus, Papyrus, 
Papyrus-Condensed, Papyrus, Papyrus-Condensed, 
Phosphate-Inline, Phosphate, Phosphate-Inline, 
Phosphate-Solid, Phosphate, Phosphate-Solid, 
PingFangHK-Light, PingFang HK, PingFangHK-Light, 
PingFangHK-Medium, PingFang HK, PingFangHK-Medium, 
PingFangHK-Regular, PingFang HK, PingFangHK-Regular, 
PingFangHK-Semibold, PingFang HK, PingFangHK-Semibold, 
PingFangHK-Thin, PingFang HK, PingFangHK-Thin, 
PingFangHK-Ultralight, PingFang HK, PingFangHK-Ultralight, 
PingFangSC-Light, PingFang SC, PingFangSC-Light, 
PingFangSC-Medium, PingFang SC, PingFangSC-Medium, 
PingFangSC-Regular, PingFang SC, PingFangSC-Regular, 
PingFangSC-Semibold, PingFang SC, PingFangSC-Semibold, 
PingFangSC-Thin, PingFang SC, PingFangSC-Thin, 
PingFangSC-Ultralight, PingFang SC, PingFangSC-Ultralight, 
PingFangTC-Light, PingFang TC, PingFangTC-Light, 
PingFangTC-Medium, PingFang TC, PingFangTC-Medium, 
PingFangTC-Regular, PingFang TC, PingFangTC-Regular, 
PingFangTC-Semibold, PingFang TC, PingFangTC-Semibold, 
PingFangTC-Thin, PingFang TC, PingFangTC-Thin, 
PingFangTC-Ultralight, PingFang TC, PingFangTC-Ultralight, 
PlantagenetCherokee, Plantagenet Cherokee, PlantagenetCherokee, 
Raanana, Raanana, Raanana, 
RaananaBold, Raanana, RaananaBold, 
SIL-Hei-Med-Jian, Hei, SIL-Hei-Med-Jian, 
SIL-Kai-Reg-Jian, Kai, SIL-Kai-Reg-Jian, 
STBaoliSC-Regular, Baoli SC, STBaoliSC-Regular, 
STBaoliTC-Regular, Baoli TC, STBaoliTC-Regular, 
STFangsong, STFangsong, STFangsong, 
STHeiti, STHeiti, STHeiti, 
STHeitiSC-Light, Heiti SC, STHeitiSC-Light, 
STHeitiSC-Medium, Heiti SC, STHeitiSC-Medium, 
STHeitiTC-Light, Heiti TC, STHeitiTC-Light, 
STHeitiTC-Medium, Heiti TC, STHeitiTC-Medium, 
STIXGeneral-Bold, STIXGeneral, STIXGeneral-Bold, 
STIXGeneral-BoldItalic, STIXGeneral, STIXGeneral-BoldItalic, 
STIXGeneral-Italic, STIXGeneral, STIXGeneral-Italic, 
STIXGeneral-Regular, STIXGeneral, STIXGeneral-Regular, 
STIXIntegralsD-Bold, STIXIntegralsD, STIXIntegralsD-Bold, 
STIXIntegralsD-Regular, STIXIntegralsD, STIXIntegralsD-Regular, 
STIXIntegralsSm-Bold, STIXIntegralsSm, STIXIntegralsSm-Bold, 
STIXIntegralsSm-Regular, STIXIntegralsSm, STIXIntegralsSm-Regular, 
STIXIntegralsUp-Bold, STIXIntegralsUp, STIXIntegralsUp-Bold, 
STIXIntegralsUp-Regular, STIXIntegralsUp, STIXIntegralsUp-Regular, 
STIXIntegralsUpD-Bold, STIXIntegralsUpD, STIXIntegralsUpD-Bold, 
STIXIntegralsUpD-Regular, STIXIntegralsUpD, STIXIntegralsUpD-Regular, 
STIXIntegralsUpSm-Bold, STIXIntegralsUpSm, STIXIntegralsUpSm-Bold, 
STIXIntegralsUpSm-Regular, STIXIntegralsUpSm, STIXIntegralsUpSm-Regular, 
STIXNonUnicode-Bold, STIXNonUnicode, STIXNonUnicode-Bold, 
STIXNonUnicode-BoldItalic, STIXNonUnicode, STIXNonUnicode-BoldItalic, 
STIXNonUnicode-Italic, STIXNonUnicode, STIXNonUnicode-Italic, 
STIXNonUnicode-Regular, STIXNonUnicode, STIXNonUnicode-Regular, 
STIXSizeFiveSym-Regular, STIXSizeFiveSym, STIXSizeFiveSym-Regular, 
STIXSizeFourSym-Bold, STIXSizeFourSym, STIXSizeFourSym-Bold, 
STIXSizeFourSym-Regular, STIXSizeFourSym, STIXSizeFourSym-Regular, 
STIXSizeOneSym-Bold, STIXSizeOneSym, STIXSizeOneSym-Bold, 
STIXSizeOneSym-Regular, STIXSizeOneSym, STIXSizeOneSym-Regular, 
STIXSizeThreeSym-Bold, STIXSizeThreeSym, STIXSizeThreeSym-Bold, 
STIXSizeThreeSym-Regular, STIXSizeThreeSym, STIXSizeThreeSym-Regular, 
STIXSizeTwoSym-Bold, STIXSizeTwoSym, STIXSizeTwoSym-Bold, 
STIXSizeTwoSym-Regular, STIXSizeTwoSym, STIXSizeTwoSym-Regular, 
STIXVariants-Bold, STIXVariants, STIXVariants-Bold, 
STIXVariants-Regular, STIXVariants, STIXVariants-Regular, 
STKaiti, STKaiti, STKaiti, 
STKaitiSC-Black, Kaiti SC, STKaitiSC-Black, 
STKaitiSC-Bold, Kaiti SC, STKaitiSC-Bold, 
STKaitiSC-Regular, Kaiti SC, STKaitiSC-Regular, 
STKaitiTC-Black, Kaiti TC, STKaitiTC-Black, 
STKaitiTC-Bold, Kaiti TC, STKaitiTC-Bold, 
STKaitiTC-Regular, Kaiti TC, STKaitiTC-Regular, 
STLibianSC-Regular, Libian SC, STLibianSC-Regular, 
STLibianTC-Regular, Libian TC, STLibianTC-Regular, 
STSong, STSong, STSong, 
STSongti-SC-Black, Songti SC, STSongti-SC-Black, 
STSongti-SC-Bold, Songti SC, STSongti-SC-Bold, 
STSongti-SC-Light, Songti SC, STSongti-SC-Light, 
STSongti-SC-Regular, Songti SC, STSongti-SC-Regular, 
STSongti-TC-Bold, Songti TC, STSongti-TC-Bold, 
STSongti-TC-Light, Songti TC, STSongti-TC-Light, 
STSongti-TC-Regular, Songti TC, STSongti-TC-Regular, 
STXihei, STHeiti, STXihei, 
STXingkaiSC-Bold, Xingkai SC, STXingkaiSC-Bold, 
STXingkaiSC-Light, Xingkai SC, STXingkaiSC-Light, 
STXingkaiTC-Bold, Xingkai TC, STXingkaiTC-Bold, 
STXingkaiTC-Light, Xingkai TC, STXingkaiTC-Light, 
STYuanti-SC-Bold, Yuanti SC, STYuanti-SC-Bold, 
STYuanti-SC-Light, Yuanti SC, STYuanti-SC-Light, 
STYuanti-SC-Regular, Yuanti SC, STYuanti-SC-Regular, 
STYuanti-TC-Bold, Yuanti TC, STYuanti-TC-Bold, 
STYuanti-TC-Light, Yuanti TC, STYuanti-TC-Light, 
STYuanti-TC-Regular, Yuanti TC, STYuanti-TC-Regular, 
Sana, Sana, Sana, 
Sathu, Sathu, Sathu, 
SavoyeLetPlain, Savoye LET, SavoyeLetPlain, 
Seravek, Seravek, Seravek, 
Seravek-Bold, Seravek, Seravek-Bold, 
Seravek-BoldItalic, Seravek, Seravek-BoldItalic, 
Seravek-ExtraLight, Seravek, Seravek-ExtraLight, 
Seravek-ExtraLightItalic, Seravek, Seravek-ExtraLightItalic, 
Seravek-Italic, Seravek, Seravek-Italic, 
Seravek-Light, Seravek, Seravek-Light, 
Seravek-LightItalic, Seravek, Seravek-LightItalic, 
Seravek-Medium, Seravek, Seravek-Medium, 
Seravek-MediumItalic, Seravek, Seravek-MediumItalic, 
ShreeDev0714, Shree Devanagari 714, ShreeDev0714, 
ShreeDev0714-Bold, Shree Devanagari 714, ShreeDev0714-Bold, 
ShreeDev0714-BoldItalic, Shree Devanagari 714, ShreeDev0714-BoldItalic, 
ShreeDev0714-Italic, Shree Devanagari 714, ShreeDev0714-Italic, 
SignPainter-HouseScript, SignPainter, SignPainter-HouseScript, 
SignPainter-HouseScriptSemibold, SignPainter, SignPainter-HouseScriptSemibold, 
Silom, Silom, Silom, 
SinhalaMN, Sinhala MN, SinhalaMN, 
SinhalaMN-Bold, Sinhala MN, SinhalaMN-Bold, 
SinhalaSangamMN, Sinhala Sangam MN, SinhalaSangamMN, 
SinhalaSangamMN-Bold, Sinhala Sangam MN, SinhalaSangamMN-Bold, 
Skia-Regular, Skia, Skia-Regular, 
Skia-Regular_Black, Skia, Skia-Regular_Black, 
Skia-Regular_Black-Condensed, Skia, Skia-Regular_Black-Condensed, 
Skia-Regular_Black-Extended, Skia, Skia-Regular_Black-Extended, 
Skia-Regular_Bold, Skia, Skia-Regular_Bold, 
Skia-Regular_Condensed, Skia, Skia-Regular_Condensed, 
Skia-Regular_Extended, Skia, Skia-Regular_Extended, 
Skia-Regular_Light, Skia, Skia-Regular_Light, 
Skia-Regular_Light-Condensed, Skia, Skia-Regular_Light-Condensed, 
Skia-Regular_Light-Extended, Skia, Skia-Regular_Light-Extended, 
SnellRoundhand, Snell Roundhand, SnellRoundhand, 
SnellRoundhand-Black, Snell Roundhand, SnellRoundhand-Black, 
SnellRoundhand-Bold, Snell Roundhand, SnellRoundhand-Bold, 
SukhumvitSet-Bold, Sukhumvit Set, SukhumvitSet-Bold, 
SukhumvitSet-Light, Sukhumvit Set, SukhumvitSet-Light, 
SukhumvitSet-Medium, Sukhumvit Set, SukhumvitSet-Medium, 
SukhumvitSet-SemiBold, Sukhumvit Set, SukhumvitSet-SemiBold, 
SukhumvitSet-Text, Sukhumvit Set, SukhumvitSet-Text, 
SukhumvitSet-Thin, Sukhumvit Set, SukhumvitSet-Thin, 
Superclarendon-Black, Superclarendon, Superclarendon-Black, 
Superclarendon-BlackItalic, Superclarendon, Superclarendon-BlackItalic, 
Superclarendon-Bold, Superclarendon, Superclarendon-Bold, 
Superclarendon-BoldItalic, Superclarendon, Superclarendon-BoldItalic, 
Superclarendon-Italic, Superclarendon, Superclarendon-Italic, 
Superclarendon-Light, Superclarendon, Superclarendon-Light, 
Superclarendon-LightItalic, Superclarendon, Superclarendon-LightItalic, 
Superclarendon-Regular, Superclarendon, Superclarendon-Regular, 
Symbol, Symbol, Symbol, 
Tahoma, Tahoma, Tahoma, 
Tahoma-Bold, Tahoma, Tahoma-Bold, 
TamilMN, Tamil MN, TamilMN, 
TamilMN-Bold, Tamil MN, TamilMN-Bold, 
TamilSangamMN, Tamil Sangam MN, TamilSangamMN, 
TamilSangamMN-Bold, Tamil Sangam MN, TamilSangamMN-Bold, 
TeluguMN, Telugu MN, TeluguMN, 
TeluguMN-Bold, Telugu MN, TeluguMN-Bold, 
TeluguSangamMN, Telugu Sangam MN, TeluguSangamMN, 
TeluguSangamMN-Bold, Telugu Sangam MN, TeluguSangamMN-Bold, 
Thonburi, Thonburi, Thonburi, 
Thonburi-Bold, Thonburi, Thonburi-Bold, 
Thonburi-Light, Thonburi, Thonburi-Light, 
Times-Bold, Times, Times-Bold, 
Times-BoldItalic, Times, Times-BoldItalic, 
Times-Italic, Times, Times-Italic, 
Times-Roman, Times, Times-Roman, 
TimesNewRomanPS-BoldItalicMT, Times New Roman, TimesNewRomanPS-BoldItalicMT, 
TimesNewRomanPS-BoldMT, Times New Roman, TimesNewRomanPS-BoldMT, 
TimesNewRomanPS-ItalicMT, Times New Roman, TimesNewRomanPS-ItalicMT, 
TimesNewRomanPSMT, Times New Roman, TimesNewRomanPSMT, 
ToppanBunkyuGothicPr6N-DB, Toppan Bunkyu Gothic, ToppanBunkyuGothicPr6N-DB, 
ToppanBunkyuGothicPr6N-Regular, Toppan Bunkyu Gothic, ToppanBunkyuGothicPr6N-Regular, 
ToppanBunkyuMidashiGothicStdN-ExtraBold, Toppan Bunkyu Midashi Gothic, ToppanBunkyuMidashiGothicStdN-ExtraBold, 
ToppanBunkyuMidashiMinchoStdN-ExtraBold, Toppan Bunkyu Midashi Mincho, ToppanBunkyuMidashiMinchoStdN-ExtraBold, 
ToppanBunkyuMinchoPr6N-Regular, Toppan Bunkyu Mincho, ToppanBunkyuMinchoPr6N-Regular, 
Trattatello, Trattatello, Trattatello, 
Trebuchet-BoldItalic, Trebuchet MS, Trebuchet-BoldItalic, 
TrebuchetMS, Trebuchet MS, TrebuchetMS, 
TrebuchetMS-Bold, Trebuchet MS, TrebuchetMS-Bold, 
TrebuchetMS-Italic, Trebuchet MS, TrebuchetMS-Italic, 
TsukuARdGothic-Bold, Tsukushi A Round Gothic, TsukuARdGothic-Bold, 
TsukuARdGothic-Regular, Tsukushi A Round Gothic, TsukuARdGothic-Regular, 
TsukuBRdGothic-Bold, Tsukushi B Round Gothic, TsukuBRdGothic-Bold, 
TsukuBRdGothic-Regular, Tsukushi B Round Gothic, TsukuBRdGothic-Regular, 
Verdana, Verdana, Verdana, 
Verdana-Bold, Verdana, Verdana-Bold, 
Verdana-BoldItalic, Verdana, Verdana-BoldItalic, 
Verdana-Italic, Verdana, Verdana-Italic, 
Waseem, Waseem, Waseem, 
WaseemLight, Waseem, WaseemLight, 
Webdings, Webdings, Webdings, 
WeibeiSC-Bold, Weibei SC, WeibeiSC-Bold, 
WeibeiTC-Bold, Weibei TC, WeibeiTC-Bold, 
Wingdings-Regular, Wingdings, Wingdings-Regular, 
Wingdings2, Wingdings 2, Wingdings2, 
Wingdings3, Wingdings 3, Wingdings3, 
YuGo-Bold, YuGothic, YuGo-Bold, 
YuGo-Medium, YuGothic, YuGo-Medium, 
YuKyo-Bold, YuKyokasho, YuKyo-Bold, 
YuKyo-Medium, YuKyokasho, YuKyo-Medium, 
YuKyo_Yoko-Bold, YuKyokasho Yoko, YuKyo_Yoko-Bold, 
YuKyo_Yoko-Medium, YuKyokasho Yoko, YuKyo_Yoko-Medium, 
YuMin-Demibold, YuMincho, YuMin-Demibold, 
YuMin-Extrabold, YuMincho, YuMin-Extrabold, 
YuMin-Medium, YuMincho, YuMin-Medium, 
YuMin_36pKn-Demibold, YuMincho +36p Kana, YuMin_36pKn-Demibold, 
YuMin_36pKn-Extrabold, YuMincho +36p Kana, YuMin_36pKn-Extrabold, 
YuMin_36pKn-Medium, YuMincho +36p Kana, YuMin_36pKn-Medium, 
YuppySC-Regular, Yuppy SC, YuppySC-Regular, 
YuppyTC-Regular, Yuppy TC, YuppyTC-Regular, 
ZapfDingbatsITC, Zapf Dingbats, ZapfDingbatsITC, 
Zapfino, Zapfino, Zapfino,

Google TF Object Detection API : SSD (Single Shot MultiBox Detector) - MobileNetV1 (with MS COCO)をローカル環境で動かす方法

機械学習 Python Tensorflow 計算機環境


f:id:yumaloop:20190119172625p:plain

1. はじめに

1.1 論文リンク

・SSDについては、リンク先のQuitta記事で原論文の日本語全訳が読めます。ありがたい。


1.2 Githubの公式ページ

・ローカル環境でSSDを動かす方法については、Githubにある公式ページRunning locallyをぜひチェックして下さい。このポストは"Running locally"の内容を噛み砕いたものです。

f:id:yumaloop:20190119173930p:plain

・Google TensorFlow API のブランチをローカルにコピー (git clone) しておくと楽です。

$ git clone https://github.com/tensorflow/models/


2. APIが要求するライブラリをローカル環境にインストールする

Tensorflow Object Detection API は以下のライブラリに依存します。
環境にない場合は随時インストールしてください。

  • Protobuf 3.0.0
  • Python-tk
  • Pillow 1.0
  • lxml
  • tf Slim (which is included in the "tensorflow/models/research/" checkout)
  • Jupyter notebook
  • Matplotlib
  • Tensorflow (>=1.9.0)
  • Cython
  • contextlib2
  • cocoapi

※ インストール方法については、下記リンクの公式ページで詳しく解説されています。
github.com



3. Githubから必要なファイルをダウンロード

3.1 Configファイル(JSON形式)のダウンロード

下記リンクから、学習済みモデル「ssd_mobilenet_v1_coco」について書かれたJSON形式の設定ファイル(.config)をローカル環境にダウンロードする。


models/ssd_mobilenet_v1_coco.config at master · tensorflow/models · GitHub

3.2 モデルの学習済み特徴量のダウンロード

下記リンク先のページから、MS COCOデータセットで学習済みのモデルの特徴量をダウンロードできる。今回はMobileNetV1の特徴量が欲しいので「COCO-trained models」から「ssd_mobilenet_v1_coco」を選択し、.tar.gz形式のファイル(.ckpt)をローカル環境にダウンロードする。


models/detection_model_zoo.md at master · tensorflow/models · GitHub

f:id:yumaloop:20190119173019p:plain

ダウンロードしたら、設定ファイル：ssd_moblienet_v1_coco.configを開き、ローカル環境でのパス(.ckptファイルがある場所)を指定する。

fine_tune_checkpoint: "PATH_TO_BE_CONFIGURED/model.ckpt"

3.3 MS COCOデータセットを読み込むためのシェルスクリプトをダウンロード

MS COCOデータセットのインポートを行うシェルスクリプト(.sh)を、下記リンクからローカル環境にダウンロードする。


models/download_and_preprocess_mscoco.sh at master · tensorflow/models · GitHub

ダウンロードしたら、設定ファイル：ssd_moblienet_v1_coco.configを開き、ローカル環境でのパス(.shファイルがある場所)を指定する。

input_path: "PATH_TO_BE_CONFIGURED/mscoco_train.record-?????-of-00100"

download_and_preprocess_mscoco.shは、元データをTensorFlowが読み込めるデータセットに変換してくれるシェルスクリプト。これを実行。(10h~15hかかる)

$ /hogehoge/hoge/download_and_preprocess_mscoco.sh


3.4 ベクトルデータからラベルと名前の紐付け（ex. 0 → "cat"）

名前の紐付けを行うファイル(.pbtxt)を、下記リンクからローカル環境にダウンロードする。


models/mscoco_label_map.pbtxt at master · tensorflow/models · GitHub

ダウンロードしたら、設定ファイル：ssd_moblienet_v1_coco.configを開き、ファイルの該当箇所にローカル環境でのパス(.pbtxtファイルがある場所)を指定する。

label_map_path: "PATH_TO_BE_CONFIGURED/mscoco_label_map.pbtxt"


3.5 実行ファイル(.py)を動かす。

　ローカル環境でSSDを動かす方法はGithubにある公式ページRunning locallyに載っている。ダウンロードしたファイルをディレクトリ構造に注意しつつ配置して、実行ファイル(.py)が動けばOK.

Tensorflow : MNISTを小規模なCNNで解いてみる

機械学習 Python Tensorflow

f:id:yumaloop:20190617180531p:plain

TensorFlowの練習がてら。
分類精度(accuracy)は98.9%

The simple implementation in python 3.6.6 with Tensorflow 1.9.0.


1.プログラム

# coding: utf-8

import tensorflow as tf
from tensorflow.contrib.learn.python.learn.datasets import mnist
from tensorflow.contrib.learn.python.learn.metric_spec import MetricSpec

learn = tf.contrib.learn
slim  = tf.contrib.slim


# モデル定義(Convolutional Neural Network)
def cnn(x, y):
    x = tf.reshape(x, [-1, 28, 28, 1])
    y = slim.one_hot_encoding(y, 10)

    #data→conv→pool→conv→pool→full→full→(softmax)→cls
    net = slim.conv2d(x,   48, [5, 5], scope = 'conv1')
    net = slim.max_pool2d(net, [2, 2], scope = 'pool1')
    net = slim.conv2d(net, 96, [5, 5], scope = 'conv2')
    net = slim.max_pool2d(net, [2, 2], scope = 'pool2')
    net = slim.flatten(net, scope = 'flatten')
    net = slim.fully_connected(net, 512, scope = 'fully_connected1')
    logits = slim.fully_connected(net, 10, activation_fn = None, scope = 'fully_connected2')
    prob = slim.softmax(logits)
    loss = slim.losses.softmax_cross_entropy(logits, y)
    train_op = slim.optimize_loss(loss, slim.get_global_step(), learning_rate = 0.001, optimizer = 'Adam')
    return {'class': tf.argmax(prob, 1), 'prob': prob}, loss, train_op


# データの読み込み
data_sets = mnist.read_data_sets('/tmp/mnist', one_hot = False)

# 変数のセット
X_train = data_sets.train.images
Y_train = data_sets.train.labels
X_test = data_sets.validation.images
Y_test = data_sets.validation.labels

# 学習ログ(validation)をコンソールに表示させる
tf.logging.set_verbosity(tf.logging.INFO)
validation_metrics = {"accuracy" : MetricSpec(metric_fn = tf.contrib.metrics.streaming_accuracy, prediction_key = "class")}
validation_monitor = learn.monitors.ValidationMonitor(X_test, Y_test, metrics = validation_metrics, every_n_steps = 100)

# 学習実行
classifier = learn.Estimator(model_fn = cnn, model_dir = '/tmp/cnn_log', config = learn.RunConfig(save_checkpoints_secs = 10))
classifier.fit(x = X_train, y = Y_train, steps = 3200, batch_size = 64, monitors = [validation_monitor])


2.ログ(コンソール画面)

ログはこんな感じ↓

...

INFO:tensorflow:global_step/sec: 2.20219
INFO:tensorflow:Starting evaluation at 2018-07-18-07:07:56
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/cnn_log/model.ckpt-3250
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2018-07-18-07:08:06
INFO:tensorflow:Saving dict for global step 3250: accuracy = 0.9898, global_step = 3250, loss = 0.03569727
INFO:tensorflow:Validation (step 3280): accuracy = 0.9898, loss = 0.03569727, global_step = 3250
INFO:tensorflow:loss = 0.0060795164, step = 3280 (40.518 sec)
INFO:tensorflow:Saving checkpoints for 3281 into /tmp/cnn_log/model.ckpt.
INFO:tensorflow:Saving checkpoints for 3316 into /tmp/cnn_log/model.ckpt.
INFO:tensorflow:Saving checkpoints for 3346 into /tmp/cnn_log/model.ckpt.
INFO:tensorflow:Saving checkpoints for 3376 into /tmp/cnn_log/model.ckpt.
INFO:tensorflow:global_step/sec: 2.33117
INFO:tensorflow:Starting evaluation at 2018-07-18-07:08:39
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/cnn_log/model.ckpt-3376
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2018-07-18-07:08:51
INFO:tensorflow:Saving dict for global step 3376: accuracy = 0.9894, global_step = 3376, loss = 0.03725804
INFO:tensorflow:Validation (step 3380): accuracy = 0.9894, loss = 0.03725804, global_step = 3376
INFO:tensorflow:loss = 0.005827286, step = 3380 (45.278 sec)
INFO:tensorflow:Saving checkpoints for 3381 into /tmp/cnn_log/model.ckpt.
INFO:tensorflow:Saving checkpoints for 3415 into /tmp/cnn_log/model.ckpt.
INFO:tensorflow:Saving checkpoints for 3449 into /tmp/cnn_log/model.ckpt.
INFO:tensorflow:Saving checkpoints for 3479 into /tmp/cnn_log/model.ckpt.
INFO:tensorflow:Loss for final step: 0.0017738211.

参考：
qiita.com

deepage.net

Stanford University CS231n: Convolutional Neural Networks for Visual Recognition

正則行列と正規行列（ムーア・ペンローズ逆行列, エルミート行列, ユニタリー行列）

数学 備忘

https://upload.wikimedia.org/wikipedia/commons/thumb/2/2f/Linear_subspaces_with_shading.svg/1200px-Linear_subspaces_with_shading.svg.png


N×Nの正方行列： {\bf A}=\left( \begin{array}{cccc} a_{11} & a_{12} & \ldots & a_{1N} \\ a_{21} & a_{22} & \ldots & a_{2N} \\ \vdots & \vdots & \ddots & \vdots \\ a_{N1} & a_{N2} & \ldots & a_{NN} \end{array} \right) について考える。


1. 正則行列


以下の命題は同値である。

※ f_{A}({\bf x})={\bf Ax} を満たす写像 f_{A}:{{\mathbb{R}}^{N}}~{\to}~{{\mathbb{R}}^{N}}を、 \mathrm{N} 次行列 {\bf A}に対応する1次変換とする。

  • 行列 {\bf A} に対して、 {\bf A}{\bf A^{-1}}~=~{\bf I} を満たす逆行列 {\bf A^{-1}}が存在する
  • 行列 {\bf A} に対して、行列式 |{\bf A}|~{\neq}~0 である。
  • 行列 {\bf A} に対して、 rank({\bf A})~=~{\mathrm{N}} である。
  • 行列 {\bf A} に対して、すべての固有値が 0 ではない。
  • 行列 {\bf A} に対して、連立1次方程式「 {\bf Ax}={\bf b}」が唯一解をもつ。
  • 行列 {\bf A} に対して、連立1次方程式「 {\bf Ax}={\bf 0}」の解は {\bf x}~=~{\bf 0} のみ。
  • 行列 {\bf A} に対して、 {\bf A} の列ベクトル ({\bf a_{1}},~{\bf a_{2}},~{\cdots},~{\bf a_{N}}) は一次独立。
  • 行列 {\bf A} に対して、 f_{A}は 「全単射」である。
  • 行列 {\bf A} に対して、 f_{A}は Imf_{A}={\mathbb{R}}^{N}をみたす。
  • 行列 {\bf A} に対して、 f_{A}は Kerf_{A}=\{ {\bf 0} \}をみたす。

2. ムーア・ペンローズ逆行列


以下の性質（１）〜（４）を満たす行列 {\bf A^{+}}を、「行列 \bf Aに対するムーア・ペンローズ逆行列」という。

（１） {\bf A}{\bf A^{+}}{\bf A}~=~{\bf A}

（２） {\bf A^{+}}{\bf A}{\bf A^{+}}~=~{\bf A^{+}}

（３） {({{\bf A^{+}}{\bf A}})}^{\mathrm{T}}~=~{\bf A^{+}}{\bf A}

（４） {({{\bf A}{\bf A^{+}}})}^{\mathrm{T}}~=~{\bf A}{\bf A^{+}}


ある行列 \bf Aに対して、（１）を満たす行列 \bf A^{+}は複数あり、これを「一般逆行列」という。
ある行列 \bf Aに対して、（１）〜（４）を満たす行列 \bf A^{+}はただ１つに定まり、これを「ムーア・ペンローズ逆行列」という。

また、 {{\bf A^{+}}{\bf A}} と {{\bf A}{\bf A^{+}}}はエルミート行列である。

3. 正規行列

以下の命題は同値である。

  • 行列 {\bf A} は正規行列である。
  • 行列 {\bf A} に対して、 {\bf A^{\mathrm{*}}}{\bf A}={\bf A}{\bf A^{\mathrm{*}}}が成り立つ。
  • 行列 {\bf A} に対して、 {\bf A} は対角化可能である。


※ 随伴行列 {\bf A}^{*}

一般に、行列 {\bf A} の随伴行列 {\bf A}^{*}は、 {\bf A}の「転置行列」を各成分において「複素共軛（実部はそのままで虚部の符号を反転する）」したものとして定義される。

なお、 {\bf A} が複素行列である場合、以下の等式が成り立つ。

　　　 {\bf A}^{*}=\overline{{\bf A}^{-1}}

さらに、 {\bf A} が実行列である場合、以下の等式が成り立つ。

　　　 {\bf A}^{*}={\bf A}^{-1}


4. エルミート行列とユニタリー行列（複素行列への拡張）

N×Nの複素正方行列： {\bf A}に対して、以下の呼び名が定義される。

  1. 行列 {\bf A} が「対称行列」である。　　　 ~{\Leftrightarrow}~~~{\bf A}={\bf A^{\mathrm{T}}}
  2. 行列 {\bf A} が「直交行列」である。　　　 ~{\Leftrightarrow}~~~{\bf A}{\bf A^{\mathrm{T}}}={\bf I}
  3. 行列 {\bf A} が「エルミート行列」である。 ~{\Leftrightarrow}~~~{\bf A}={\bf A^{\mathrm{*}}}
  4. 行列 {\bf A} が「ユニタリー行列」である。 ~{\Leftrightarrow}~~~{\bf A}{\bf A^{\mathrm{*}}}={\bf I}


ここで、「対称行列」「直交行列」「エルミート行列」「ユニタリー行列」はいずれも正規行列であり、対角化可能となる。
エルミート行列は、対称位置にある成分が互いに複素共役になっている。

https://mathwords.net/wp-content/uploads/2016/09/erumiito-300x163.png

5. まとめ


◯「逆行列」　　(→正方行列である前提を拡張すると...)　「ムーア・ペンローズ逆行列」

◯「対称行列」　(→実行列である前提を拡張すると...)　　「エルミート行列」
　
◯「直交行列」　(→実行列である前提を拡張すると...)　　「ユニタリー行列」

参考：

mathwords.net

随伴行列 - Wikipedia