Wednesday, 25 July 2012

Amarok StatSyncing GSoC: weeks 8 and 9

Yes, I'm still working on my GSoC project about statistics synchronization in Amarok. I've somehow skipped the last report (I was too deeply immersed in coding), to this one will be a bit more beefy. In short, I've fixed some bugs of the EngineController (the thing actually responsible for playback) that negatively affected Last.fm scrobbling first and then I've rewrote the class that actually scrobbles the tracks.
Little side-effect of Last.fm improvements: correction suggestions
What I've done in week 8:
  • The rather big patch series that fixed data-loss bugs was finally merged to master to be included in Amarok 2.6 release candidate. Please give it a good testing.
  • Improved EngineController and Phonon interaction with regards to meta-data changes.
  • Improved MetaStream classes that represent audio streams. They now implement statistics (and some other fields) so that you can for example see how many songs you've listened to on particular stream. The statistics are forgotten on Amarok exit though.
  • Improved detection of "track finished playing" in EngineController. This fixes the "play count increased by 2" bug and may fix other issues with weird playlist playback.
  • Above 3 fixes (and a little more) are included in a review request that I intend to merge into master for 2.6 release candidate and in enginecontroller-fixes branch of my Amarok repository clone. This should be the last thing blocking the RC.
What I've done the last week:
  • Rebased my gsoc  branch on top of liblastfm1 which is Harald Sitter's work on porting Amarok to liblastfm >= 1.0. I need some its features and Amarok 2.7 will depend on it anyway.
  • Made EngineController detect some meta-data changes in streams and interpret them as song changes. This makes scrobbling possible even when listening to an infinite stream.
  • Cleaned up some Amarok code interfacing Last.fm to do it in more clean way.
  • Rewrote ScrobblerAdapter (responsible for scrobbling) to use all the new EngineController goodness. This allowed to ditch all the hacky code that tracked currently playing song by itself. This is also a preparation to splitting scrobbling code into controller/scrobbling service, which will allow to plug in different scrobbling back-ends (not just Last.fm) in future.
  • Fixed the MutiTrack API. Many streams (for example ones in the Internet category of the Media Sources pane) use it and it was a bit broken yielding broken meta-data updates, track-advancing, scrobbling etc.
  • Above fixes are published in my usual gsoc review request and are to be found in gsoc branch of my repository.
Problems I've faced:
  • EngineController code is convoluted. The worst thing is that there is a lot of code for special cases (multi-playback, multi-source, bounded-playback tracks) that is used infrequently. That code tends to break as it is little tested and sometimes not updated to reflect newest changes.
  • EngineController has to deal with different phonon back-ends and this is delicate. Phonon-vlc likes to emit aboutToFinish() twice in some cases while phonon-gstreamer likes to emit currentSourceChanged() twice and metaDataChanged() even more often (witch no reason). EngineController has to be written very carefully not to explode in case of such rough handling.
What's next:
  • Continuing the work on Last.fm scrobbling and synchronization. I'm a little bit behind the schedule with it currently, but the way is paved now.
  • I'll need to push some changes to liblastfm to make everything planned possible. The upstream is however pretty alive so this shouldn't be a big problem.
You can test my work by pulling and building gsoc branch of my Amarok git repository clone, it already works quite well! I also publish weekly diffs with more technical details on KDE's review board which may be a more convenient way to review my code and to comment on it: week 1 week 2 week 3 week 4 (week 5 done in →) week 6 week 7 (master bugfix) week 8 (master bugfix) week 9

Tuesday, 10 July 2012

Amarok StatSyncing GSoC: week 7 aka Where are all my statistics? And where is Amarok 2.6?

Yes, I'm still working on my GSoC project about statistics synchronization in Amarok. Past week was a bit unusual, because I've worked mainly on fixing bugs in Amarok and not directly on statistics synchronization. I think it is justifiable, because a) the bugs are about data loss which includes statistics b) they block the 2.6 release c) my GSoC timeline allows me to do so as items for week 7 are already implemented d) the data loss bugged me for long time. :-)

What I've done this week:
  • Fixed bug 292245 where statistics, lyrics and labels of a song are lost if its meta-data are changed in Amarok and then it is moved. The fix is already pushed to master, enjoy!
  • Fixed bug 298275 where statistics of all tracks are lost in some cases such as toggling Local Files & USB Mass Storage Backend in Configuration -> Plugins (don't do it!). This turned out to be rather beefy patch series that currently awaits review (hi Ralf!) as it touches some delicate parts of Amarok.
  • Made SqlScanResultProcessor clean up the mess left by the Organize Files functionality (bug 289338), included in above series.
  • (Hopefully) made Amarok much more cautious when it comes to destructive operations in order not to lose precious data such as ratings when something goes wrong. For example, no entries are removed from the database if a serious error occurs before. Included in above series.
Problems I've faced:
  • SqlScanResultProcessor was rather hard to understand. I spend hours just to realise what's going on.
  • Debugging collection scanner is time consuming. I've performed countless code → build → restore database → test cycles.
What's next:
  • I'll file usability review request about GUI of statistics synchronization.
  • I'll move on to the second part of my project - Last.fm integration in statistics synchronization.
Fortunately, I finally succeeded and all data loss bugs I was able to reproduce are fixed now. This means that the Amarok 2.6 is closer now because both main fixed bugs were marked as release blockers. On the other hand, 2 bugs still remain and our release guy, Bart, is time-constrained currently (fortunately he documented the whole process well).

You can test my work by pulling and building gsoc branch of my Amarok git repository clone, it already works quite well! I also publish weekly diffs with more technical details on KDE's review board which may be a more convenient way to review my code and to comment on it: week 1 week 2 week 3 week 4 (week 5 done in →) week 6 week 7 (master bugfix)

Monday, 2 July 2012

Amarok StatSyncing GSoC: week 6

Yes, I'm still working on my GSoC project about statistics synchronization in Amarok. I've skipped one report, because last week I was busy with my university exam and real life. This week is another story, I've added synchronization preparation page (so that you can check what to sync), implemented automatic non-interactive synchronization and more.
New dialog this week: preparation for synchronization.
What I've done this week:
  • Factored Meta::Track::recentPlayCount() into capability, it didn't have much sense.
  • Added above page for the first step of the synchronization, added Back button to the second page.
  • Made Amarok remember syncing preferences: what fields and collections to synchronize. Local Collection and iPod collections are checked by default.
  • Implemented automatic unattended synchronization: checked (or enabled by default not-unchecked) collections will be synchronized automagically as soon as they appear or on start-up (with some delay to let them settle). If ratings conflicts are detected, it will fall-back to interactive mode and you can make the background job interactive as well.
  • Improved minor visual things (ratings are now nicely painted using stars) plus did some code clean-up so that first part of my GSoC approaches merge-ready state.
Problems I've faced:
  • I've fought with QTableView and sizeHint()s a bit, but I've won. :-)
  • The dialogs probably need usability and design review, sure I'll contact KDE Usability group before merging.
What's next:
  • According to my schedule, I've fulfilled week 5 and week 7 with this (funny). More importantly, first part of my project is roughly done by this week's advancements, and I'm quite happy with the results. I think that the unattended synchronization is particularly useful: you just tell Amarok what to synchronize once and it does everything else for you; that's what 99% users will be using most of the time in my opinion.
  • This week I'll finally start looking at Last.fm things.
You can test my work by pulling and building gsoc branch of my Amarok git repository clone, it already works quite well! I also publish weekly diffs with more technical details on KDE's review board which may be a more convenient way to review my code and to comment on it: week 1 week 2 week 3 week 4 week 6