Implement History Manager

[Alecaddd]

Let's kick-off the conversation on how to implement an History Manager to handle undo and redo actions.

Memory management
The first approach is going the standard route of implementing a memory based history manager, and giving users the ability to define the memory usage limit.

Version control
The other approach we could consider is to save the various actions in a local history file if the user hasn't saved the current project, or save it directly inside the file in case it exists.
This will give us the flexibility to have a potential infinite history without taxing on memory management.
If we decide to go this route, we will need to implement a manual "clear file history" for the user in order to better control the file size. Even adding an option like "Clear file history on shutdown" might be useful.

Thoughts?

[albfan]

You can do in any way you want. Serialization branch should allow to store the canvas state and object properties at every moment, so we just need to add signals to trigger new history steps.

[danirabbit]

I know at one point there was some discussion around using libgit for versioning. Since SVG is text it seems like using an existing and very robust and optimized system would be good.

This would also make it easy to share and collaborate on Akira projects if they’re just git repos. You could get history saved even if you’re not necessarily running Akira.

I wonder if some kind of system could be worked out so you get every commit from the last 24 hours and then commits each day for a week and then weekly commits for a month etc. So that you still get fairly granular history and it just gets slightly less detailed with time

[Alecaddd]

Yes, the libgit is most likely gonna be used to store the various states of a saved file (whenever the user hits CTRL+s), and offer a built-in version control inside the .akira file.
I guess we could extend that to incorporate history management, so a your project comes with both all the saved versions and the full history.

I guess we could downfall to memory based history if the user didn't save the file yet.

[danirabbit]

It’s [current year], why design an app with manual saving :)

[Alecaddd]

I understand the argument, and I agree for the majority of the cases, but for a design application I wouldn't implement that type of automation out of the box.
The main reasons are:

• Don't automatically save a new file in a location not decided by the user.
• Don't prompt the user with a save dialog when a new project starts.
• Don't save changes that might be temporary (eg. open a file, delete an element to export something quickly, close the file without saving because the change shouldn't be permanent.)
• Don't save a new file when not necessary (eg. create a file to trim an image and export it as smaller PNG, close it without saving the file since the user doesn't need that as a project.)

I might consider adding an option like "Autosave the file at every change", but I prefer to have it as opt-in and not opt-out, especially if no initial file has been saved.

I'm more than happy to have my opinions crushed and my ideas changed, but only if we can find solutions that don't lock the users into a workflow they might not want.

[Imerion]

I agree with above post: auto-saving can cause problems in many situations and I don't like when software do it. There is also the issue of reducing battery time on laptops by spinning up disks when saving is not needed.

[JasonKraft]

Hey, I had some thoughts on how history management could work for Akira. I feel like there are some hybrid approaches not yet considered that could provide a best-of-both-worlds solution. It sounds like between this issue and #90 there are several features desired by users:

1. Users want a way to undo/redo atomic operations (e.g. rotating, scaling, coloring canvas items)
2. Users want a way to undo/redo large chunks of changes (i.e. checking out/reverting git commits)
3. Users don't want to worry about losing their work if the program crashes, but they also do not want a pure auto-save feature.

Here's my 2 cents on how I would implement a solution for all three user requirements:

Maintain an in-memory stack of edit operations for undo/redo

This is fairly straightforward. Each atomic operation/action would be defined as a class inheriting from a common interface/abstract class (e.g. IInvertableOperation). Whenever someone makes a change to a canvas item, that would be placed on an in-memory stack. To undo that change, you simply pop the operation off the top of the stack and call it's undo() implementation.

In reality, I'd imagine this may be better implemented as a doubly-linked list, moving up and down the list as the user calls undo/redo. If a user then does undo and makes a new edit without calling redo, every operation in the list past your current pointer would be deleted and the new operation would be appended.

Whenever a user hits save, serialize both the current project state and the undo/redo stack

The project state can continue to be saved as a JSON file, and the undo/redo stack can be stored as a separate JSON file as an array. By saving both, the user can commit changes to the local git repo without having to worry about losing the ability to undo atomic operations for each commit. Then in effect:

• Each commit would represent a manual save
• The undo/redo stack would represent the list of changes in chronological order between saves

To avoid accidental data loss, regularly automatically save in a separate branch of the git repo

Whenever the user manually chooses to save, this branch can then be merged into the master branch and deleted. The next time the program starts, if it detects the backup branch still exists it should act as though the program crashed and use the backup branch to reload lost changes.

Let me know if that aligns with what everyone was going for. I'd be happy to try to mock up some UML class diagrams for the undo/redo feature when I get some free time and possibly start to take a stab at it in code (though I've never worked w/ Vala before, so it might be a learning curve).

[Imerion]

That sounds like a smart solution to me, perhaps with an option to limit how far undo/redo history is saved to avoid getting too large files and using too much memory. Something like saving the last 50 or last 100 actions, depending on the setting.

[jdittrich]

I guess it makes sense to first get standard undo/redo in place.
Just like @JasonKraft, I would also suggest an in-memory-stack of invertable actions (see e.g. draw2D implementation )

[jdittrich]

Once undo/redo is in place, consider versioning.

There seem to be several established options or interfaces for this. I personally like googleDoc’s/Etherpads approach, where you can "star" and name states of the history (sadly not retrospectively, but this should be no problem, actually)

gDocs sidebar-style:

Etherpads time-slider:

Both (gDocs, etherpad), however, use operational state transforms where the whole history is kept, anyway.
However, a flag-this-for-versioning UI could also work with approaches where the history is not kept or is only kept to a limit and then "baked" into one fully stored state without history.

[Alecaddd]

I'm gonna list some ideas for a potential technical implementation, which are mostly related to the work proposed in #531

Serialize all the things

I like it and I think it's the way to go.
Having a static method we can call when we need to serialize an item or the entire Canvas is good.
That will give us the flexibility to "keep" an item(s) in a very tiny and manageable string, which consequentially will allow us to:

• Duplicate an item.
• Create a Component/Symbol which affects any linked/reference item.
• Store the entire Canvas in a Git commit and easily show users the diff in a visual representation.

Undo/Redo with the entire Canvas serialization

I'm not sure about this.
It worries me about performance as we're clearing the entire Canvas and reloading all serialized items at every undo or redo.
Maybe it won't affect anything and once we solve the potential performance issues with goocanvas, it might not matter much, but it still feels an overkill.

What my half backed idea was...

Every action that modifies something (translation, resize, change color, item creation, item deletion, etc.) would cause the saving of that specific action in the HistoryManager with the initial state of the selected items.
When we then register a keyup or mouse up event, we save the changed state of those selected items.
If the serialized attributes are identical, we ignore the action and don't save that state.

An action would be a serialization of all the currently selected items in an Object, which will have a label representing the triggered actions.
Roughly, something like this:

{
  action: "Moved items",
  items: {
    item1: {
      old: {..serialized item's attributes...},
      new: {..serialized item's attributes...},
    },
    item2: {
      old: {..serialized item's attributes...},
      new: {..serialized item's attributes...},
    },
    item3: {
      old: {..serialized item's attributes...},
      new: {..serialized item's attributes...},
    },
    ....
  }
 } 

With item1, item2, etc, being the item ID.
Right now items don't have a unique ID, but that should be fixed and we should have a method in place to quickly grab an item by its ID.

Storing each "action" in a Json Object allows us to visually return to the user the full history of changes, and when an undo happens we can cherry pick and alter only the affected items.

[AshishS-1123]

Using Version Control every time you save a file seems a little too much.
For example consider the following scenario- Lets say you add an image of around 10 MB in size and save your work. This image will be saved in your version control. Then you decide that you don't want that image so you remove it. But because of version control, it will forever be present in your history and over time adding more images will bloat your design file.
Plus, a lot of people have the habit of frequently saving their work, which will create unnecessary commits in the history that will be difficult to navigate.

So instead of using Git on every save, we create a button that the user can press to create a commit only when they are comfortable with their changes. This will give the user some flexibility over what they want in their history.

As for Undo/Redo, we take a little help from Vim (the text editor). When you open a file using Vim, all the changes you make are stored in a .swp file. This file can be used for performing undo and redo operations, for recovering data in case of failure and to prevent the file from being edited by multiple people at the same time. This file is created only when you start the application and destroyed when you save and close it.

As an analogy consider how software developers write code. When we make changes to a code file, the IDE provides the undo/redo features, not Git. And only after creating meaningful changes do we use Git to save those changes in out repository.
Imagine the nightmare if Git decided to store all our keystrokes !

[Imerion]

Agreed on this! The way I work my files would become gigantic and I'd much prefer smaller files. A "Save with History" button would be a smart alternative. Then it could be used when you've reached certain milestones, etc.

[jdittrich]

Using Version Control every time you save a file seems a little too much.

Indeed.

Why not go with the standard way of doing this, using a command pattern and a history stack?

Having that in place we can still use a vim-like swap file (e.g. in case the app crashes), serialize, store diffs etc. but basic (multiple) undo/redo already works in a way both familiar to many devs and to users alike.

[albfan]

Test first before do claims about a version control system.

That image will be a packed in same pack of it is exactly the same, so there's no overhead to add remove indefinitely. If image is not exactly de same, yes you have to store all them.

What's an undo manager that requires you to save safe points? That's only a sophisticated version of you storing your Akira project as foo.akira, foo1.akira... foo-01-01-2021.akira.

In general do not improve performance as a design. Performance and speed comes after design.

[Alecaddd]

Using Version Control every time you save a file seems a little too much.
For example consider the following scenario- Lets say you add an image of around 10 MB in size and save your work. This image will be saved in your version control. Then you decide that you don't want that image so you remove it. But because of version control, it will forever be present in your history and over time adding more images will bloat your design file.

I considered this scenario when thinking about this feature as what you wrote was a common issue of the past versions of Sketch.
One important aspect I probably forgot to mention is the ability to "clean up" the saved file history.
Exposing a simple and straightforward option to remove unnecessary old versions will allow users to slim down a file.

Plus, a lot of people have the habit of frequently saving their work, which will create unnecessary commits in the history that will be difficult to navigate.

This is a non issue as we won't create any commits if the user hits "Save" hundreds of times and nothing changes.
Also, the commit only pushes the diff, not the whole changesets since the beginning of the file.

So instead of using Git on every save, we create a button that the user can press to create a commit only when they are comfortable with their changes. This will give the user some flexibility over what they want in their history.

That's risky as the software should have smart defaults that help the user to avoid mistakes.
Not saving the file history automatically and relying on the user's action is too risky.

Regarding the undo/redo, we're currently working on Lib2 which it's implementing a serialize/deserialize approach to history, and based on initial tests, undoing a 1000 items creation all at once is extremely fast and doesn't create any performance issue.