tk_nn_classifier.classifiers package

Submodules

tk_nn_classifier.classifiers.graph_selector module

class tk_nn_classifier.classifiers.graph_selector.GraphSelector(config, embedding)

Bases: object

add_graph(input, training_mode, embedding_initializer)

tk_nn_classifier.classifiers.spacy_classifier module

class tk_nn_classifier.classifiers.spacy_classifier.SpacyClassifier(config)

Bases: tk_nn_classifier.classifiers.base_classifier.BaseClassifier

build_and_train()

build_graph()

evaluate(dataset, mode='train', losses=0)

evaluate_on_tests()

load_data()

load_saved_model(model_path=None)

predict_batch(texts)

prediction_on_set(dataset)

process_with_saved_model(input)

save(output_dir)

train(train_data, eval_data)

tk_nn_classifier.classifiers.tf_best_export module

class tk_nn_classifier.classifiers.tf_best_export.BestCheckpointsExporter(name='best_exporter', serving_input_receiver_fn=None, event_file_pattern='eval/*.tfevents.*', compare_fn=<function _loss_smaller>, assets_extra=None, as_text=False, exports_to_keep=5)

Bases: tensorflow_estimator.python.estimator.exporter.BestExporter

export(estimator, export_path, checkpoint_path, eval_result, is_the_final_export)

Exports the given Estimator to a specific format.

Args:

estimator: the Estimator to export. export_path: A string containing a directory where to write the export. checkpoint_path: The checkpoint path to export. eval_result: The output of Estimator.evaluate on this checkpoint. is_the_final_export: This boolean is True when this is an export in the

end of training. It is False for the intermediate exports during the training. When passing Exporter to tf.estimator.train_and_evaluate is_the_final_export is always False if TrainSpec.max_steps is None.

Returns:

The string path to the exported directory or None if export is skipped.

tk_nn_classifier.classifiers.tf_classifier module

class tk_nn_classifier.classifiers.tf_classifier.TFClassifier(config)

Bases: tk_nn_classifier.classifiers.base_classifier.BaseClassifier

build_and_train()

build_graph()

evaluate_on_tests()

input_fn(data_path, shuffle_and_repeat=False)

load_data_set(data_path)

load_embedding()

load_saved_model(model_path=None)

model_fn(features, labels, mode, params)

predict_batch(data_path)

predict_on_text(text)

process_with_saved_model(input)

static serving_input_receiver_fn()

serving input, to work with tensorflow estimator command tools like: saved_model_cli, also for prediction input

train()

tk_nn_classifier.classifiers.tf_multi_feat_classifier module

class tk_nn_classifier.classifiers.tf_multi_feat_classifier.TFMultiFeatClassifier(config)

Bases: tk_nn_classifier.classifiers.base_classifier.BaseClassifier

build_and_train()

build_graph()

evaluate_on_tests()

input_fn(data_path, shuffle_and_repeat=False)

load_data_set(data_path)

load_embedding()

load_saved_model(model_path=None)

model_fn(features, labels, mode, params)

predict_batch(data_path)

predict_on_text(text)

process_with_saved_model(input)

static serving_input_receiver_fn(max_sequence_length)

input shape:

input_0: [batch_size, max_sequence_length[0] ] input_1: [batch_size, max_sequence_length[1] ] …… len: [batch_size, number_of_input ]

train()

tk_nn_classifier.classifiers.utils module

class tk_nn_classifier.classifiers.utils.ConfusionMatrix(eval, gold)

Bases: object

Generate a confusion matrix for multiple classification

params:

• eval: a list of integers or strings of predicted classes
• gold: a list of integers or strings of known classes

output:

• confusion_matrix: 2-dimensional list of pairwise counts

class tk_nn_classifier.classifiers.utils.FileHelper

Bases: object

static last_modified_folder(model_path)

class tk_nn_classifier.classifiers.utils.TrainHelper

Bases: object

static accuracy(eval, gold)

static max_dict_value(cats_dicts)

static print_progress(loss, accu)

static print_progress_header()

static print_test_result(eval, gold)

tk_nn_classifier.classifiers.utils.creation_date(path_to_file)

Try to get the date that a file was created, falling back to when it was last modified if that isn’t possible. See http://stackoverflow.com/a/39501288/1709587 for explanation.

tk_nn_classifier.classifiers.utils.eval_predictions(predictions, gold_labels)

input:

• predictions
• gold_labels

func:

• compute the accuracy, and print out
• compute the precision and recall, and print out

output:

• accuracy
• precision
• recall

Module contents

The classifiers in different packages

class tk_nn_classifier.classifiers.SpacyClassifier(config)

Bases: tk_nn_classifier.classifiers.base_classifier.BaseClassifier

build_and_train()

build_graph()

evaluate(dataset, mode='train', losses=0)

evaluate_on_tests()

load_data()

load_saved_model(model_path=None)

predict_batch(texts)

prediction_on_set(dataset)

process_with_saved_model(input)

save(output_dir)

train(train_data, eval_data)

class tk_nn_classifier.classifiers.TFClassifier(config)

Bases: tk_nn_classifier.classifiers.base_classifier.BaseClassifier

build_and_train()

build_graph()

evaluate_on_tests()

input_fn(data_path, shuffle_and_repeat=False)

load_data_set(data_path)

load_embedding()

load_saved_model(model_path=None)

model_fn(features, labels, mode, params)

predict_batch(data_path)

predict_on_text(text)

process_with_saved_model(input)

static serving_input_receiver_fn()

serving input, to work with tensorflow estimator command tools like: saved_model_cli, also for prediction input

train()

class tk_nn_classifier.classifiers.KerasClassifier(config)

Bases: tk_nn_classifier.classifiers.base_classifier.BaseClassifier

build_and_train()

build_graph()

A multi-layer cnn

params:

• input_dimension: the dimension of the input data
• l_rate: the learning rate

output: neural netword model

evaluate(test_file)

Evaluate on the data set

evaluate_on_tests()

load_data_set(data_path)

load_embedding()

load_saved_model(model_path=None)

predict_on_text(input)

prepare_train_eval_data()

process_with_saved_model(input)

train(train_data, eval_data)

Training process

class tk_nn_classifier.classifiers.TFMultiFeatClassifier(config)

Bases: tk_nn_classifier.classifiers.base_classifier.BaseClassifier

build_and_train()

build_graph()

evaluate_on_tests()

input_fn(data_path, shuffle_and_repeat=False)

load_data_set(data_path)

load_embedding()

load_saved_model(model_path=None)

model_fn(features, labels, mode, params)

predict_batch(data_path)

predict_on_text(text)

process_with_saved_model(input)

static serving_input_receiver_fn(max_sequence_length)

input shape:

input_0: [batch_size, max_sequence_length[0] ] input_1: [batch_size, max_sequence_length[1] ] …… len: [batch_size, number_of_input ]

train()