Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

fix block.export #17970

Merged
merged 9 commits into from
Sep 1, 2020
Merged
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 13 additions & 4 deletions python/mxnet/gluon/block.py
Original file line number Diff line number Diff line change
Expand Up @@ -1234,7 +1234,7 @@ def infer_type(self, *args):
"""Infers data type of Parameters from inputs."""
self._infer_attrs('infer_type', 'dtype', *args)

def export(self, path, epoch=0, remove_amp_cast=True):
def export(self, path, epoch=0, remove_amp_cast=True, allow_extra=True):
"""Export HybridBlock to json format that can be loaded by
`gluon.SymbolBlock.imports`, `mxnet.mod.Module` or the C++ interface.

Expand All @@ -1248,7 +1248,11 @@ def export(self, path, epoch=0, remove_amp_cast=True):
will be created, where xxxx is the 4 digits epoch number.
epoch : int
Epoch number of saved model.

remove_amp_cast : bool, optional
Whether to remove the amp_cast and amp_multicast operators, before exporting the model.
allow_extra : bool, optional
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the use-case to save parameters that are not used by the symbol? Do we need this option?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not necessary.

Whether to save extra parameters whose names are not in the result symbol.
User can set allow_extra to True to load these parameters with old mxnet.mod.Module.set_params API.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mxnet.mod.Module.set_params does not exist anymore.

Returns
-------
symbol_filename : str
Expand All @@ -1271,8 +1275,13 @@ def export(self, path, epoch=0, remove_amp_cast=True):
if name in arg_names:
arg_dict['arg:%s'%name] = param._reduce()
else:
assert name in aux_names
arg_dict['aux:%s'%name] = param._reduce()
if name.endswith('running_mean') or name.endswith('running_var') \
or name.endswith('moving_mean') or name.endswith('moving_var'):
assert name in aux_names
arg_dict['aux:%s'%name] = param._reduce()
Copy link
Contributor

@leezu leezu Aug 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do we ensure that no aux parameter with a name different to running_mean, running_var, etc is missing from the resulting params file if allow_extra is False?

else:
if allow_extra:
arg_dict['aux:%s'%name] = param._reduce()
save_fn = _mx_npx.save if is_np_array() else ndarray.save
params_filename = '%s-%04d.params'%(path, epoch)
save_fn(params_filename, arg_dict)
Expand Down