何事西风不待人

A couple of months ago, I opened my Firefox as usual, I found out that when I click on the address bar, it dared to select everything in it as if any creepy browser would do! The first thing I did is to check the clickSelectsAll pref -- what I always do first after getting a new install of Firefox on Windows is setting this pref to false, and setting doubleClickSelectsAll pref to true: that is the default behaviour for Firefox on GNU/Linux... that time at least...

Much to my disappointment, the pref are setting correctly, but nothing works. I tried to search for this, but without success, maybe since this bug was so new, or I was to silly to apply the right keywords (huh, maybe the latter one?). So I created an account on Mozilla's bugtracker just to open a bug. At the same time I switched back to the ESR version of Firefox, which rid me (till recently) of this stupid address bar but still granted me security updates. A few days ago, however, Firefox ESR was unfortunately eventually shipped with this bug, calling for a solution.

Your work is gonna make Krita significantly different. -- Wolthera, Krita developer and digital artist

Krita's undo system, namely kritacommand, was added 8 years ago to Calligra under the name of kundo2, as a fork of Qt's undo framework. The use of undo commands, however, might have an even longer history. Undo commands provide a way to revert individual actions. Up to now, most (though not all) undo commands do it by providing two sets of code that do and undo the actions, respectively. Drawbacks of this system includes (1) it is not very easy to manage; (2) it may introduce duplicated code; and (3) it makes it hard to access a previous document state without actually going back to that state. What I do is to start getting rid of such situation.

The plan for a new system is to use shallow copies to store documents at different states. Dmitry said "it was something we really want to do and allows us to make historical brushes (fetch content from earlier document states)." And according to him, he spent years to implement copy-on-write on paint layers. He suggested me to start from vector layers which he thought would be easier since it does not need to be very thread-safe.

I completely understood that was a challenge, but did not realize where the difficult part was until I come here. Copy-on-write is not the challenging part. We have QSharedDataPointer and almost all the work is to routinely replace the same code. Porting tools is more difficult. The old flake tools are running under the GUI thread, which makes no requirement on thread-safety. Technically we do not need to run it in a stroke / in image thread but with no multithreading the tools runs too slowly on some computers (read as "my Thinkpad laptop") so I am not unwilling to take this extra challenge. In previous posts I described how the strokes work and the problems I encountered. Besides that there are still some problems I need to face.

the HACK code in the stroke strategy

At the last of the strokes post, I proposed a fix to the crash when deleting KisNode, which is messy. After testing with Dmitry at the sprint, we discovered that the real problems lies in KoShapeManager's updateTreeCompressor. It is used to schedule updates of its R-tree. However, it is run at the beginning of every other operation so Dmitry says it is no longer needed. After the compressor was removed we are safe to delete the node normally so there would be no need for such hack code.

Path tool crashing when editing calligraphic shapes

Calligraphic shapes, coming from Karbon, is a shape created by hand-drawing. It has many path points and editing it using path tool usually leads to a crash. Dmitry tested it with ASan and discovered the problem occurs because the path points, which is fetched in the GUI thread to paint the canvas, could be deleted when editing the shape. He suggests to apply a lock to the canvas, not allowing the image and GUI threads to access the shapes concurrently.

Keeping selections after undo/redoing

This challenge is a smaller one. The shape selections were not kept, since they are not part of the layer. It was owned by the layer's shape manager, though, but a cloned layer would take a brand-new shape manager. In addition undo() and redo() will now replace the whole layer, so pointers to original shapes are no longer valid. This means merely keeping the selections from the shape manager would not work. The solution is to map the selected shapes to the cloned layer, which would be kept in the undo command. The strategy I use is similar to what we have done for layers: go through the whole heirarchy of the old layer and push everything into a queue; go through the heirarchy of the cloned layer in the same order and each time take the first shape in the queue; if the popped shape is in the selection, we add its counterpart in the cloned layer to our new selection.

For now the tools should be working and the merge request is prepared for final review. Hopefully it would make its way to master soon.

Hi -)) haven't posted for some time, because I was busy travelling and coding for the first half of the month. From Aug 5 to Aug 9, I went to the Krita Sprint in Deventer, Netherlands.

According to Boud, I was the first person to arrive. My flight took a transit via Hong Kong where some flights were affected due to natural and social factors, but fortunately mine was not one of them. Upon arrival in Amsterdam I got a ticket for the Intercity to Deventer. Railway constructions made me take a transfer via Utrecht Centraal, but that was not a problem at all: the station has escalators going both up to the hall, and down to the platforms (in China you can only go to the hall by stairs or elevator (which is often crowded after you get off)). When I got out of Deventer Station, Boud immediately recognized me (how?!). It was early in the morning, and the street's quietness was broken by the sound of me dragging my suitcase. Boud led me through Deventer's crooked streets and alleys to his house.

For the next two days people gradually arrived. I met my main mentor Dmitry (magician!) and his tiger, Sagoskatt, which I (and many others) have mistaken for a giraffe. He was even the voice actor for Sago. He had got quite a lot of insights into the code base (according to Boud, "80%") and solved a number of bugs in Krita (but he said he introduced a lot of bugs, ha!). Also I met David Revoy (my favourite painter!), the author of Pepper and Carrot. And Tiar, our developer who started to work full-time on Krita this year; she had always been volunteering to support other Krita users and always on the IRC and Reddit. And two of other three GSoC students for the year: Blackbeard (just as his face) and Hellozee. Sh_zam could not come and lost communications due to political issues, which was really unfortunate (eh at least now he can be connected). It is feels so good to be able to see so many people in the community -- they are so nice! And it is such an experience to hack in a basement church.

On Aug 7 we went to the Open Air Museum. It displays a large extent of the history in the Netherlands, how their people lived. After a really delicious lunch we went out and started to do paintings. I was to paint on my Surface using Krita, but unfortunately it went out of battery so I had to gave up and painted on a postcard. The tram in the museum is my favourite one (I am always fond of transit) and they even have a carhouse where stood lots of old vehicles. Except for my head which hit the ceiling of the coach three times, everything that day was wonderful.

The next day was the main meeting. In the morning we discussed the development plans for Krita. Bugs. Stability. New features. David Revoy came up again with the docker size problem, which Boud simply called it "a Qt problem." He said, "Yes I do know what to do with that, but new users probably don't and thus we gotta address it and not solely blame Qt." (Yeah it troubled me a lot as well!) Another thing closely related to me was building on Windows, which was largely neglected by KDE. In the afternoon the focus shifted to marketing. I did not know much about it, but it is a fact that we cannot produce electricity out of love. We spent quite a lot of time on the painting competition for Krita. Where it should be held. How to collect the paintings. How to filter out good pictures. Krita promotes new artists. They promote our software.

For the next two days people started leaving. I left on the 10th, and then slept for a whole day when I got to Nanjing (so tired...). On Aug 14th I left again for Toronto, and then restarted to write code and debug. I finally got the time to write this post today, as I finally fixed a crash in my project. It is almost finished, and soon another post would be made on it.

Okay, good news today. I have been porting DefaultTool to the new node-replacing system and it is working now, finally, at least for the part I have already done.

The work involves combining a number of different modules in Krita: the stroke system, KoInteractionTool and its interaction strategies, and, well, the COW mechanism in Flake.

KoInteractionTool is the class used to manage the interaction with vector shapes, and is subclassed by DefaultTool. The behaviours of KoInteractionTool (and thus DefaultTool) are defined by KoInteractionStrategys. Upon the press of the mouse button, DefaultTool creates an instance of some subclass of KoInteractionStrategy, say, ShapeMoveStrategy, according to the point of the click as well as keyboard modifiers. Mouse move events after that are all handled by the interaction strategy. When the mouse is released, the interaction strategy's finishInteraction() is called, and then createCommand(). If the latter returns some KUndo2Command, the command is added to the undo history. Till now it sounds simple.

So how does the stroke system come in? I have experimented the interaction strategy without the stroke system (https://invent.kde.org/tusooaw/krita/commit/638bfcd84c622d3cfefda1e5132380439dd3fdc2), but it is really slow and even freezes Krita for a while sometimes. The stroke system allows the modification of the shapes to run in the image thread, instead of the GUI thread. A stroke is a set of jobs scheduled and run by a KisStrokesFacade (here, KisImage). One creates the stroke in a strokes facade using a stroke strategy, which defines the behaviour of the stroke. After creation, jobs can be added to the stroke and then executed at some later time (it is asynchronous).

So combining these two, we have an interaction strategy and a stroke strategy -- when the interaction strategy is created, we start the stroke in the image; when there is mouse move, we add individual jobs that change the shapes to the stroke; when the mouse released, we end the stroke. My discussion with Dmitry firstly tended to make the interaction strategy inherit the stroke strategy but later it proves not a viable solution since the interaction strategy is owned and deleted by KoInteractionTool while the stroke strategy is owned by the stroke --- which will lead to double deletion. So we divide it into two classes instead: the interaction strategy starts the stroke, and the stroke strategy takes a copy of the current active layer upon creation; when handling mouse move events, a job is added to the stroke to modify the current layer; finally when the interaction finishes, the interaction strategy ends the stroke and creates an undo command if the layer has been changed.

A problem I found lies in the final stage--if the mouse is released as soon as being pressed and no undo command is created, Krita will simply crash. It does not happen when I use gdb to start Krita so it seems to be a timing issue though it leads to difficulty for debugging as well. Dmitry used a self-modified version of Qt to produce a backtrace, indicating the problem probably lies in KisCanvas2's canvasUpdateCompressor, which is not thread-safe. However, after I changed it to KisThreadSafeSignalCompressor, the crash still happens, unfortunately.

The final inspiration comes from the comments in KisThreadSafeSignalCompressor, though. It indicates we cannot delete the compressor from other threads --- we have to use obj->deleteLater() instead, since it lies in the gui thread. And aha, that is the problem. The stroke strategy's destructor is executed in the image thread; if the undo command is not created, there is only one reference to our copied KisNode, namely in our stroke strategy, so it has to be destructed there. However, upon the creation of the KisNode, it is moved into the gui thread. So it simply means we cannot let it be deleted in the image thread. The solution looks a little bit messy, but it works:

1
2
3
4
5

KisNode *node = m_d->originalState.data(); // take the address from KisSharedPtr
node->ref(); // prevent KisSharedPtr from deleting the node
m_d->originalState.clear(); // now node is not being referenced by any KisSharedPtr
node->deref(); // the reference count is now zero again
node->deleteLater(); // it will be deleted by the event loop, later

At the end of June I finished copy-on-write vector layers. From the very beginning, I have been researching into possibilities to make kritaflake implicitly sharable. In that post I mentioned the way Sean Parent uses for Photoshop, and adapted it for the derived d-pointers in Flake.

Derived d-pointers

TL;DR: We got rid of it.

As I mentioned in the task page, derived d-pointers originally in Flake are a barrier to implicit sharing. One of the reasons is that we need to write more code (either KisSharedDescendent wrapper class, or repeated code for virtual clone functions). Also, derived d-pointers do not actually encapsulate the data in the parent classes -- for example, the members in KoShapePrivate are all accessible by descendents of KoShape, say, KoShapeContainer. That is probably not how encapsulating should work. So in the end we decided to get rid of derived d-pointers in Flake.

This leads to one problem, however, in the class KoShapeGroup. KoShapeGroup is a descendent of KoShapeContainer, which owns a KoShapeContainerModel that can be subclassed to control the behaviour when a child is added to or removed from the container. KoShapeGroup uses ShapeGroupContainerModel which performs additional operations specific to KoShapeGroup.

After I merged my branch into master, it was said that Flake tests failed under address sanitizer (ASan). I took a look and discovered that there was use after free in the class KoShapeGroup, namely the use of its d-pointer. The use is called by the destructor of KoShapeContainer, which calls KoShapeContainerModel::deleteOwnedShapes(), which removes individual shapes from the container, which then calls KoShapeGroup::invalidateSizeCache(). The original situation was:

1. destructor of KoShapeGroup was called;
2. members defined in KoShapeGroup got deleted (nothing, because everything is in the derived d-pointer which is defined in KoShape);
3. destructor of KoShapeContainer was called, which calls d->model->deleteOwnedShapes();
4. then that of KoShape, which deletes all the private members.

But after the derived d-pointers are converted to normal ones, the calling sequence upon destruction becomes:

1. destructor of KoShapeGroup was called;
2. members defined in KoShapeGroup got deleted (its own d-pointer);
3. destructor of KoShapeContainer was called, which calls d->model->deleteOwnedShapes();
4. d->model is a ShapeGroupContainerModel, which will call KoShapeGroup::invalidateSizeCache();
5. that last function accesses the d-pointer of KoShapeGroup, USE AFTER FREE.

In order to solve this problem we have to manually call model()->deleteOwnedShapes() in the destructor of KoShapeGroup, at which time the d-pointer is still accessible.

q-pointers

TL;DR: We also got rid of it.

q-pointers are a method used in Qt to hide private methods from the header files, in order to improve binary compatibility. q-pointers are stored in *Private classes (ds), indicating the object that owns this private instance. But this is, of course, conflicting with the principle of "sharing" because the situation now is that multiple objects can own the same data. The q-pointers in flake is rather confusing under such circumstances, since the private data cannot know which object is the caller.

To avoid this confusion, there are multiple ways:

1. to move all the functions regarding q-pointers to the public classes;
2. to pass the q-pointer every time when calling those functions in private classes; or
3. to add another layer of "shared data" in the d-pointer and keep the q-pointers in the unshared part.

implicit sharing

To enable implicit sharing for the KoShape hierarchy, the only thing left to be done is to change the QScopedPointer<Private> d; in the header file to QSharedDataPointer<Private> d; and make the private classes inherit QSharedData. This step is rather easy and then just run the tests to make sure it does not break anything. Horray!

Over the past few weeks I have been working on the Snapshot Docker, and now it is finished already. -))

The idea of snapshots is to make copies of the current document and allow users to return to them at a later time. This is a part of my whole Google Summer of Code project, which aims to bring Krita a better undo/redo system. When fully implemented, it will fully replace the current mechanism that stores actions with one that stores different states. That is to say, Krita will create a snapshot of the document for every undoable step.

Snapshot Docker is not only a feature requested by artists but also a experimental implementation of the clone-replace mechanism. It has the following key parts:

1. Cloning the document, which is provided by KisDocument::lockAndCloneForSaving(), which is already implemented in master.

2. Replace the current document by another one, which is previously cloned.

Part (1) is already implemented so the work falls mainly on Part (2). My original approach is to replace the document and image pointers in KisView and KisCanvas, but it is not viable since other parts of the program have signal/slot connections on the KisDocument and KisImage, and directly replacing the two pointers will not only fail to work but also cause weird crashes. After discussing with Dmitry, we find out that it is probably better not to touch these two pointers, but to replace the content within KisDocument and KisImage. It is therefore suggested that two member functions be made, namely KisDocument::copyFromDocument and KisImage::copyFromImage. These functions copies data from another document/image to the current one, avoiding the changes to the pointers inside the original instance. Eh, except for the nodes, since we have to reset and refresh the nodes in the image.

It is also important to notify other parts of Krita about the change in the document. One important thing is tell the layer docker about the changes in the nodes (they are completely different), which is done using the KisImage::sigLayersChangedAsync() signal. The current activated node is also stored and restored, by using the strategy of linearizing the layer tree using a queue, and then finding the corresponding node in the cloned image. Note that when restoring, we are unable to find layer by uuid, since they should change when copied to the current image (the comments in KisImage says the only situation where we should keep the uuids is for saving).

Another interesting thing is the palettes. Krita 4.2.0 allows documents to store their own, local palettes. The palette list is but a QList<KoColorSet *>, meaning that only creating a new QList of the same pointers will not work. This is because, the palettes are controlled by canvas resource manager, which takes the responsibility to delete them. Therefore, when taking snapshots, we had better take deep copies of the KoColorSets. And then another problem comes: the snapshots own their KoColorSets because they are not controlled by the resource manager in any way; but the KisDocument in the view does not. So we have to set up another flag, ownsPaletteList, to tell the document whether it should delete the palettes in the destructor.

And now the work has shifted to the refactoring of kritaflake, the library that mainly handles vector layers and shapes. I converted the whole KoShape hierarchy to implicit sharing where possible, but some tests are broken. I am now on Windows, where unit tests do not run. I will continue the development of flake as soon as I get access to my Linux laptop.