Add possibility to merge several classes to dataset scripts #156
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vierja
suggested changes
Feb 15, 2018
Now when there are merged classes the class.json file, which is used for labeling predicted videos/pics, will be empty. This fixes a problem in which all predictions generated from a dataset that had merged classes would be labeled with the label of the first class. Eg.: If you generated a dataset based on coco that filtered all classes except car, bus, and truck, and that also merged these classes into a single class. The predictions generated by predict.py or the web server would be labeled as car, when in fact they could've been a truck or a bus too. The idea behind completely removing the label instead of picking a new one is that a model that was trained to predict a single type of object, would portray this information more globally, instead of on an per object basis. For example in the name of the .jpg or .mpg file it created, or something like this. Still, there are probably use cases in which it would be useful to let the user pick the label. This could be added in the future after choosing a good console argument name for this parameter, and seeing if we could somehow merge it with the --merge argument while also maintaining the ability to have the label be empty.
nagitsu
suggested changes
Feb 20, 2018
| @@ -67,16 +68,18 @@ def transform(dataset_reader, data_dir, output_dir, splits, only_classes, | |||
| # All splits must have a consistent set of classes. | |||
| classes = None | |||
|
|
|||
| merge_classes = merge_classes in ('True', 'true', 'TRUE') | |||
nagitsu
reviewed
Feb 20, 2018
| @@ -30,16 +30,17 @@ def get_output_subfolder(only_classes, only_images, limit_examples, | |||
| @click.option('--data-dir', help='Where to locate the original data.') | |||
| @click.option('--output-dir', help='Where to save the transformed data.') | |||
| @click.option('splits', '--split', required=True, multiple=True, help='Which splits to transform.') # noqa | |||
| @click.option('--only-classes', help='Whitelist of classes.') | |||
| @click.option('--only-classes', multiple=True, help='Whitelist of classes.') | |||
| @click.option('--merge-classes', help='Merge all classes into a single class') | |||
There was a problem hiding this comment.
Can't we name it --single-class or --merge-all to make the fact that only a single class creatd more explicit?
nagitsu
reviewed
Feb 20, 2018
| json.dump(self._reader.classes, tf.gfile.GFile(classes_file, 'w')) | ||
| if self._reader.merge_classes: | ||
| # Don't assign a name to the class if its a merge of several others | ||
| json.dump([''], tf.gfile.GFile(classes_file, 'w')) |
There was a problem hiding this comment.
Have an option to set the class name before merging the pull request.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds the ability to merge classes with the
--merge-classesoption. Very useful when used with--only-classesto create datasets for detecting certain type of objects. For example, a dataset for detecting 4 wheeled vehicles that mergescar,busandtruckfrom the coco dataset.Could also be useful without the
--only-classesoption to train the network to behave as a sort of more discriminative RPN.