-
Notifications
You must be signed in to change notification settings - Fork 43
SystemError: <built-in function imread> returned NULL without setting an error #32
Description
while training the model using this command:
python -m nmt --src=gloss --tgt=de --train_prefix=../Data/phoenix2014T.train --dev_prefix=../Data/phoenix2014T.dev --test_prefix=../Data/phoenix2014T.test --out_dir=/home/ml2/nslt-master/nslt/model --vocab_prefix=../Data/phoenix2014T.vocab --source_reverse=True --num_units=1000 --num_layers=4 --num_train_steps=150000 --residual=True --attention=luong --base_gpu=0 --unit_type=gru
The error below was raised. Do you know how should I fix it ?
log_file=/home/ml2/nslt-master/nslt/model/log_1616265448
created train model with fresh parameters, time 2.03s
created infer model with fresh parameters, time 0.50s
288
Traceback (most recent call last):
File "/home/ml2/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call
return fn(*args)
File "/home/ml2/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1306, in _run_fn
status, run_metadata)
File "/usr/lib/python3.6/contextlib.py", line 88, in exit
next(self.gen)
File "/home/ml2/.local/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.UnknownError: SystemError: returned NULL without setting an error
[[Node: PyFunc = PyFunc[Tin=[DT_STRING, DT_BOOL], Tout=[DT_FLOAT], token="pyfunc_5"](arg0, PyFunc/input_1)]]
[[Node: IteratorGetNext = IteratorGetNextoutput_shapes=[[300,227,227,3], [1]], output_types=[DT_FLOAT, DT_INT32], _device="/job:localhost/replica:0/task:0/cpu:0"]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/ml2/nslt-master/nslt/nmt.py", line 378, in
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/home/ml2/.local/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/ml2/nslt-master/nslt/nmt.py", line 368, in main
run_main(FLAGS, default_hparams, train_fn, inference_fn)
File "/home/ml2/nslt-master/nslt/nmt.py", line 361, in run_main
train_fn(hparams, target_session=target_session)
File "/home/ml2/nslt-master/nslt/train.py", line 265, in train
run_full_eval(model_dir, infer_model, infer_sess, eval_model, eval_sess, hparams, summary_writer, sample_src_data, sample_tgt_data)
File "/home/ml2/nslt-master/nslt/train.py", line 188, in run_full_eval
run_sample_decode(infer_model, infer_sess, model_dir, hparams, summary_writer, sample_src_data, sample_tgt_data)
File "/home/ml2/nslt-master/nslt/train.py", line 135, in run_sample_decode
_sample_decode(loaded_infer_model, global_step, infer_sess, hparams, infer_model.iterator, src_data, tgt_data, infer_model.src_placeholder, summary_writer)
File "/home/ml2/nslt-master/nslt/train.py", line 418, in _sample_decode
nmt_outputs, attention_summary = model.decode(sess)
File "/home/ml2/nslt-master/nslt/model.py", line 409, in decode
_, infer_summary, _, sample_words = self.infer(sess)
File "/home/ml2/nslt-master/nslt/model.py", line 397, in infer
return sess.run([self.infer_logits, self.infer_summary, self.sample_id, self.sample_words])
File "/home/ml2/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File "/home/ml2/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1124, in _run
feed_dict_tensor, options, run_metadata)
File "/home/ml2/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
options, run_metadata)
File "/home/ml2/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnknownError: SystemError: returned NULL without setting an error
[[Node: PyFunc = PyFunc[Tin=[DT_STRING, DT_BOOL], Tout=[DT_FLOAT], token="pyfunc_5"](arg0, PyFunc/input_1)]]
[[Node: IteratorGetNext = IteratorGetNextoutput_shapes=[[300,227,227,3], [1]], output_types=[DT_FLOAT, DT_INT32], _device="/job:localhost/replica:0/task:0/cpu:0"]]"