Notes on PinePhone and mobile GNU/Linux systems 关于 PinePhone 和移动端 GNU/Linux 系统的笔记
From a weird Dolphin bug 从一个奇怪的 Dolphin bug 说起
Some multilingual hacks on Hexo 瞎折腾Hexo的多语言
Fix firefox address bar clickSelectsAll bug (Spoiler: no re-compilation required.)
A couple of months ago, I opened my Firefox as usual, I found out that
when I click on the address bar, it dared to select everything in it
as if any creepy browser would do! The first thing I did is to check
the clickSelectsAll
pref -- what I always do first after getting a
new install of Firefox on Windows is setting this pref to false
,
and setting doubleClickSelectsAll
pref to true
: that is the default
behaviour for Firefox on GNU/Linux... that time at least...
Much to my disappointment, the pref are setting correctly, but nothing works. I tried to search for this, but without success, maybe since this bug was so new, or I was to silly to apply the right keywords (huh, maybe the latter one?). So I created an account on Mozilla's bugtracker just to open a bug. At the same time I switched back to the ESR version of Firefox, which rid me (till recently) of this stupid address bar but still granted me security updates. A few days ago, however, Firefox ESR was unfortunately eventually shipped with this bug, calling for a solution.
Pay another respect to kritacommand--which we are going beyond
Your work is gonna make Krita significantly different. -- Wolthera, Krita developer and digital artist
Krita's undo system, namely kritacommand
, was added 8 years ago to Calligra under the name of kundo2
, as a fork of Qt's undo framework. The use of undo commands, however, might have an even longer history. Undo commands provide a way to revert individual actions. Up to now, most (though not all) undo commands do it by providing two sets of code that do and undo the actions, respectively. Drawbacks of this system includes (1) it is not very easy to manage; (2) it may introduce duplicated code; and (3) it makes it hard to access a previous document state without actually going back to that state. What I do is to start getting rid of such situation.
The plan for a new system is to use shallow copies to store documents at different states. Dmitry said "it was something we really want to do and allows us to make historical brushes (fetch content from earlier document states)." And according to him, he spent years to implement copy-on-write on paint layers. He suggested me to start from vector layers which he thought would be easier since it does not need to be very thread-safe.
I completely understood that was a challenge, but did not realize where the difficult part was until I come here. Copy-on-write is not the challenging part. We have QSharedDataPointer
and almost all the work is to routinely replace the same code. Porting tools is more difficult. The old flake tools are running under the GUI thread, which makes no requirement on thread-safety. Technically we do not need to run it in a stroke / in image thread but with no multithreading the tools runs too slowly on some computers (read as "my Thinkpad laptop") so I am not unwilling to take this extra challenge. In previous posts I described how the strokes work and the problems I encountered. Besides that there are still some problems I need to face.
the HACK code in the stroke strategy
At the last of the strokes post, I proposed a fix to the crash when deleting KisNode
, which is messy. After testing with Dmitry at the sprint, we discovered that the real problems lies in KoShapeManager
's updateTreeCompressor
. It is used to schedule updates of its R-tree. However, it is run at the beginning of every other operation so Dmitry says it is no longer needed. After the compressor was removed we are safe to delete the node normally so there would be no need for such hack code.
Path tool crashing when editing calligraphic shapes
Calligraphic shapes, coming from Karbon, is a shape created by hand-drawing. It has many path points and editing it using path tool usually leads to a crash. Dmitry tested it with ASan and discovered the problem occurs because the path points, which is fetched in the GUI thread to paint the canvas, could be deleted when editing the shape. He suggests to apply a lock to the canvas, not allowing the image and GUI threads to access the shapes concurrently.
Keeping selections after undo/redoing
This challenge is a smaller one. The shape selections were not kept, since they are not part of the layer. It was owned by the layer's shape manager, though, but a cloned layer would take a brand-new shape manager. In addition undo()
and redo()
will now replace the whole layer, so pointers to original shapes are no longer valid. This means merely keeping the selections from the shape manager would not work. The solution is to map the selected shapes to the cloned layer, which would be kept in the undo command. The strategy I use is similar to what we have done for layers: go through the whole heirarchy of the old layer and push everything into a queue; go through the heirarchy of the cloned layer in the same order and each time take the first shape in the queue; if the popped shape is in the selection, we add its counterpart in the cloned layer to our new selection.
For now the tools should be working and the merge request is prepared for final review. Hopefully it would make its way to master
soon.
The Sprint
Hi -)) haven't posted for some time, because I was busy travelling and coding for the first half of the month. From Aug 5 to Aug 9, I went to the Krita Sprint in Deventer, Netherlands.
According to Boud, I was the first person to arrive. My flight took a transit via Hong Kong where some flights were affected due to natural and social factors, but fortunately mine was not one of them. Upon arrival in Amsterdam I got a ticket for the Intercity to Deventer. Railway constructions made me take a transfer via Utrecht Centraal, but that was not a problem at all: the station has escalators going both up to the hall, and down to the platforms (in China you can only go to the hall by stairs or elevator (which is often crowded after you get off)). When I got out of Deventer Station, Boud immediately recognized me (how?!). It was early in the morning, and the street's quietness was broken by the sound of me dragging my suitcase. Boud led me through Deventer's crooked streets and alleys to his house.
For the next two days people gradually arrived. I met my main mentor Dmitry (magician!) and his tiger, Sagoskatt, which I (and many others) have mistaken for a giraffe. He was even the voice actor for Sago. He had got quite a lot of insights into the code base (according to Boud, "80%") and solved a number of bugs in Krita (but he said he introduced a lot of bugs, ha!). Also I met David Revoy (my favourite painter!), the author of Pepper and Carrot. And Tiar, our developer who started to work full-time on Krita this year; she had always been volunteering to support other Krita users and always on the IRC and Reddit. And two of other three GSoC students for the year: Blackbeard (just as his face) and Hellozee. Sh_zam could not come and lost communications due to political issues, which was really unfortunate (eh at least now he can be connected). It is feels so good to be able to see so many people in the community -- they are so nice! And it is such an experience to hack in a basement church.
On Aug 7 we went to the Open Air Museum. It displays a large extent of the history in the Netherlands, how their people lived. After a really delicious lunch we went out and started to do paintings. I was to paint on my Surface using Krita, but unfortunately it went out of battery so I had to gave up and painted on a postcard. The tram in the museum is my favourite one (I am always fond of transit) and they even have a carhouse where stood lots of old vehicles. Except for my head which hit the ceiling of the coach three times, everything that day was wonderful.
The next day was the main meeting. In the morning we discussed the development plans for Krita. Bugs. Stability. New features. David Revoy came up again with the docker size problem, which Boud simply called it "a Qt problem." He said, "Yes I do know what to do with that, but new users probably don't and thus we gotta address it and not solely blame Qt." (Yeah it troubled me a lot as well!) Another thing closely related to me was building on Windows, which was largely neglected by KDE. In the afternoon the focus shifted to marketing. I did not know much about it, but it is a fact that we cannot produce electricity out of love. We spent quite a lot of time on the painting competition for Krita. Where it should be held. How to collect the paintings. How to filter out good pictures. Krita promotes new artists. They promote our software.
For the next two days people started leaving. I left on the 10th, and then slept for a whole day when I got to Nanjing (so tired...). On Aug 14th I left again for Toronto, and then restarted to write code and debug. I finally got the time to write this post today, as I finally fixed a crash in my project. It is almost finished, and soon another post would be made on it.
Strokes are Working Now
Okay, good news today. I have been porting DefaultTool to the new node-replacing system and it is working now, finally, at least for the part I have already done.
The work involves combining a number of different modules in Krita: the stroke system, KoInteractionTool and its interaction strategies, and, well, the COW mechanism in Flake.
KoInteractionTool
is the class used to manage the interaction with vector shapes, and
is subclassed by DefaultTool
. The behaviours of KoInteractionTool
(and thus DefaultTool
)
are defined by KoInteractionStrategy
s. Upon the press of the mouse button, DefaultTool
creates an instance of some subclass of KoInteractionStrategy
, say, ShapeMoveStrategy
,
according to the point of the click as well as keyboard modifiers. Mouse move events after that
are all handled by the interaction strategy. When the mouse is released, the interaction strategy's
finishInteraction()
is called, and then createCommand()
. If the latter returns some
KUndo2Command
, the command is added to the undo history. Till now it sounds simple.
So how does the stroke system come in? I have experimented the interaction strategy
without the stroke system (https://invent.kde.org/tusooaw/krita/commit/638bfcd84c622d3cfefda1e5132380439dd3fdc2),
but it is really slow and even freezes Krita for a while sometimes. The stroke system
allows the modification of the shapes to run in the image thread, instead of the GUI thread.
A stroke is a set of jobs scheduled and run by a KisStrokesFacade
(here, KisImage
).
One creates the stroke in a strokes facade using a stroke strategy, which defines the behaviour
of the stroke. After creation, jobs can be added to the stroke and then executed at some later
time (it is asynchronous).
So combining these two, we have an interaction strategy and a stroke strategy --
when the interaction strategy is created, we start the stroke in the image;
when there is mouse move, we add individual jobs that change the shapes to the stroke;
when the mouse released, we end the stroke.
My discussion with Dmitry firstly tended to make the interaction strategy inherit
the stroke strategy but later it proves not a viable solution since the interaction
strategy is owned and deleted by KoInteractionTool
while the stroke strategy is owned
by the stroke --- which will lead to double deletion. So we divide it into two classes
instead: the interaction strategy starts the stroke, and the stroke strategy takes a copy
of the current active layer upon creation; when handling mouse move events, a job is added
to the stroke to modify the current layer; finally when the interaction finishes,
the interaction strategy ends the stroke and creates an undo command if the layer has been
changed.
A problem I found lies in the final stage--if the mouse is released as soon as being pressed
and no undo command is created, Krita will simply crash. It does not happen when I use gdb
to start Krita so it seems to be a timing issue though it leads to difficulty for debugging as
well. Dmitry used a self-modified version of Qt to produce a backtrace, indicating the problem
probably lies in KisCanvas2
's canvasUpdateCompressor
, which is not thread-safe. However,
after I changed it to KisThreadSafeSignalCompressor
, the crash still happens, unfortunately.
The final inspiration comes from the comments in KisThreadSafeSignalCompressor
, though. It
indicates we cannot delete the compressor from other threads --- we have to use obj->deleteLater()
instead, since it lies in the gui thread. And aha, that is the problem. The stroke strategy's destructor
is executed in the image thread; if the undo command is not created, there is only one reference to
our copied KisNode
, namely in our stroke strategy, so it has to be destructed there. However, upon
the creation of the KisNode
, it is moved into the gui thread. So it simply means we cannot let it
be deleted in the image thread. The solution looks a little bit messy, but it works:
1 | KisNode *node = m_d->originalState.data(); // take the address from KisSharedPtr |
`make -j5 kritaflake`
At the end of June I finished copy-on-write vector layers. From the very beginning, I have been
researching into
possibilities to make kritaflake
implicitly sharable. In that post I mentioned the way
Sean Parent uses for Photoshop, and adapted it for the derived d-pointers in Flake.
Derived d-pointers
TL;DR: We got rid of it.
As I mentioned in the task page, derived d-pointers originally
in Flake are a barrier to implicit sharing. One of the reasons is that we need to write more code (either
KisSharedDescendent
wrapper class, or repeated code for virtual clone functions). Also, derived
d-pointers do not actually encapsulate the data in the parent classes -- for example, the members in
KoShapePrivate
are all accessible by descendents of KoShape
, say, KoShapeContainer
. That is probably
not how encapsulating should work. So in the end we decided to get rid of derived d-pointers in Flake.
This leads to one problem, however, in the class KoShapeGroup
. KoShapeGroup
is a descendent of KoShapeContainer
,
which owns a KoShapeContainerModel
that can be subclassed to control the behaviour when a child is added to or
removed from the container. KoShapeGroup
uses ShapeGroupContainerModel
which performs additional operations
specific to KoShapeGroup
.
After I merged my branch into master, it was said that Flake tests failed under address sanitizer (ASan). I
took a look and discovered that there was use after free in the class KoShapeGroup
, namely the use of its
d-pointer. The use is called by the destructor of KoShapeContainer
, which calls
KoShapeContainerModel::deleteOwnedShapes()
, which removes individual shapes
from the container, which then calls KoShapeGroup::invalidateSizeCache()
. The original situation was:
- destructor of
KoShapeGroup
was called; - members defined in
KoShapeGroup
got deleted (nothing, because everything is in the derived d-pointer which is defined inKoShape
); - destructor of
KoShapeContainer
was called, which callsd->model->deleteOwnedShapes()
; - then that of
KoShape
, which deletes all the private members.
But after the derived d-pointers are converted to normal ones, the calling sequence upon destruction becomes:
- destructor of
KoShapeGroup
was called; - members defined in
KoShapeGroup
got deleted (its own d-pointer); - destructor of
KoShapeContainer
was called, which callsd->model->deleteOwnedShapes()
; d->model
is aShapeGroupContainerModel
, which will callKoShapeGroup::invalidateSizeCache()
;- that last function accesses the d-pointer of
KoShapeGroup
, USE AFTER FREE.
In order to solve this problem we have to manually call model()->deleteOwnedShapes()
in the destructor
of KoShapeGroup
, at which time the d-pointer is still accessible.
q-pointers
TL;DR: We also got rid of it.
q-pointers are a method used in Qt to hide private methods from the header files, in order to improve
binary compatibility. q-pointers are stored in *Private classes (d
s), indicating the object that owns
this private instance. But this is, of course, conflicting with the principle of "sharing" because
the situation now is that multiple objects can own the same data. The q-pointers in flake is rather confusing
under such circumstances, since the private data cannot know which object is the caller.
To avoid this confusion, there are multiple ways:
- to move all the functions regarding q-pointers to the public classes;
- to pass the q-pointer every time when calling those functions in private classes; or
- to add another layer of "shared data" in the d-pointer and keep the q-pointers in the unshared part.
implicit sharing
To enable implicit sharing for the KoShape
hierarchy, the only thing left to be done is to
change the QScopedPointer<Private> d;
in the header file to QSharedDataPointer<Private> d;
and make the private classes inherit QSharedData
. This step is rather easy and then just run the
tests to make sure it does not break anything. Horray!
Snapshot Docker
Over the past few weeks I have been working on the Snapshot Docker, and now it is finished already. -))
The idea of snapshots is to make copies of the current document and allow users to return to them at a later time. This is a part of my whole Google Summer of Code project, which aims to bring Krita a better undo/redo system. When fully implemented, it will fully replace the current mechanism that stores actions with one that stores different states. That is to say, Krita will create a snapshot of the document for every undoable step.
Snapshot Docker is not only a feature requested by artists but also a experimental implementation of the clone-replace mechanism. It has the following key parts:
Cloning the document, which is provided by
KisDocument::lockAndCloneForSaving()
, which is already implemented in master.Replace the current document by another one, which is previously cloned.
Part (1) is already implemented so the work falls mainly on Part (2). My original approach is to replace the
document and image pointers in KisView
and KisCanvas
, but it is not viable since other parts of the program
have signal/slot connections on the KisDocument
and KisImage
, and directly replacing the two pointers will
not only fail to work but also cause weird crashes. After discussing with Dmitry,
we find out that it is probably better not to touch these two pointers, but to replace the content within
KisDocument
and KisImage
. It is therefore suggested that two member functions be made, namely
KisDocument::copyFromDocument
and KisImage::copyFromImage
. These functions copies data from another document/image
to the current one, avoiding the changes to the pointers inside the original instance. Eh, except for the nodes,
since we have to reset and refresh the nodes in the image.
It is also important to notify other parts of Krita about the change in the document. One important thing is
tell the layer docker about the changes in the nodes (they are completely different), which is done using the
KisImage::sigLayersChangedAsync()
signal. The current activated node is also stored and restored, by using
the strategy of linearizing the layer tree using a queue, and then finding the corresponding node in the cloned
image. Note that when restoring, we are unable to find layer by uuid, since they should change when copied to
the current image (the comments in KisImage
says the only situation where we should keep the uuids is for saving).
Another interesting thing is the palettes. Krita 4.2.0 allows documents to store their own, local palettes.
The palette list is but a QList<KoColorSet *>
, meaning that only creating a new QList
of the same pointers
will not work. This is because, the palettes are controlled by canvas resource manager, which takes the responsibility
to delete them. Therefore, when taking snapshots, we had better take deep copies of the KoColorSet
s. And then
another problem comes: the snapshots own their KoColorSet
s because they are not controlled by the resource manager
in any way; but the KisDocument
in the view does not. So we have to set up another flag, ownsPaletteList
, to
tell the document whether it should delete the palettes in the destructor.
And now the work has shifted to the refactoring of kritaflake
, the library that mainly handles vector layers and
shapes. I converted the whole KoShape
hierarchy to implicit sharing where possible, but some tests are broken. I
am now on Windows, where unit tests do not run. I will continue the development of flake as soon as I get access to
my Linux laptop.