Options
QAP not responding while updating "Frequent" DB
- QAP not responding while updating "Frequent" DB
-
QAP not responding while updating "Frequent" DB
LaurentG > 2020-08-15 12:07
Hi Jean,
I got again issues of QAP not responding, or more precisely, responding very very slowly.
I already had the problem in the past, and we already discussed it.... but with no solution.
Since it re-occured these days, I have a little bit investigated, and I think I have (at least partially) understood the problem (and I hope you'll understand how to fix it
).
Actually, it occurs while QAP is udating the QAP_Frequent.DB... that takes a long time ((typically up to 20 or 30 seconds) when database is "relatively" big.
And since the update takes such a long time, there is serious probability that we try to use QAP during this time... which makes the problem visible.
How did I run my test :
1) I first change the parameter UsageDbIntervalSeconds to a small value (let's say 10 sec) to force frequent update.
2) Thanks to QAP, I open a folder not opened for a long time
3) I reopen QAP menu immediately after, and the folder I just opened in 2) is not (yet) in "Recent Folders"
4) I wait just a little bit more than the duration set in 1), let's say 12 seconds
5) I try again to open QAP menu.... and it's unsuccessfully : QAP is not responding. I repeat trying to open the menu until it opens.
6) Next time it opens, "Recent Folders" contains folder open in 2), and we can see in explorer that database file has just been updated.
If I stop QAP / delete the database / restart QAP, a new DB is created , a lot smaller, and repeating the process above, DB update is fast enough to never make QAP unresponding. But of course, I have lost "Frequent folders" and "Frequent files" (but "Recent" ones are still there....)
Several remarks now :
- When I say "big" databse, this one was around +/- 1Mb, +/- 2500 lines in Usage table
- I tried to "compact" the database thanks to "DB Browser for SQlite" application : no advantage
- I tried also to remove oldest lines in Usage table, reducing the database approx. by half, but this didn't change anything to the problem.
- Actually, it is not feasible to put a value less than 60 seconds to UsageDbIntervalSeconds in the GUI, but feasible updating manually QuickAccessPopup.ini file. I agree that such a small value is surely not a good idea in normal life, but it helps a lot to run the process above without waiting too long between steps...
- When QAP is not responding, it is not only the display of the menu that is not responding : right click on QAP's icon in tray do not opens the "system" menu...
- I tried to set the DiagMode in QuickAccessPopup.ini, but almost nothing is logged in QuickAccessPopup-DIAG.txt (contrary to previous versions of QAP)
And finally one "suggestion" : Is it the same thread that updates database and manages the menu ? If yes, wouldn't it be feasible to have Db update "delegated" to a separate thread ? -
RE: QAP not responding while updating "Frequent" DB
Jean Lalonde > 2020-08-15 14:53
Hi Laurent,
I'll start with remarks and, at the end with better tool to understand what happens.
> Actually, it occurs while QAP is udating the QAP_Frequent.DB... that takes a long time ((typically up to 20 or 30 seconds) when database is "relatively" big.
> And since the update takes such a long time, there is serious probability that we try to use QAP during this time... which makes the problem visible.
Following my experience, I am surprised that the QAP_Frequent.DB takes more that 1 second. In my test, it averages at 150-200 ms. I think there is something else to look for. And in the SQL database world, QAP database size (by default limited to 3 megs) is very tiny.
> - Actually, it is not feasible to put a value less than 60 seconds to UsageDbIntervalSeconds in the GUI, but feasible updating manually QuickAccessPopup.ini file. I agree that such a small value is surely not a good idea in normal life, but it helps a lot to run the process above without waiting too long between steps...
You are right. Doing it for debugging is OK but 60 secs+ is safer in normal usage.
> I tried to set the DiagMode in QuickAccessPopup.ini, but almost nothing is logged in QuickAccessPopup-DIAG.txt (contrary to previous versions of QAP)
Normal, debugging content vary depending on what I need to debug.
> And finally one "suggestion" : Is it the same thread that updates database and manages the menu ? If yes, wouldn't it be feasible to have Db update "delegated" to a separate thread ?
Database refresh is done in a separate thread with lower priority. It should not interfere with menu opening.
> - When QAP is not responding, it is not only the display of the menu that is not responding : right click on QAP's icon in tray do not opens the "system" menu...
Maybe this is not related to the QAP database but to the "Scheduled QAP Menu refresh" in "Options, Advanced Options" (by default, this option is off). If you turned it on before, try disabling it and see if this makes a change? Or keep it on and enable the Beep (just under) to see ("hear") how long the refresh takes. In my personal menu, it takes around 2-3 seconds and during those 2-3 seconds (every 5 minutes by default), the QAP menu is not available.
![[Image: y0URuGk.png]](https://i.imgur.com/y0URuGk.png)
-
RE: QAP not responding while updating "Frequent" DB
LaurentG > 2020-08-16 12:29
Bonjour Jean,
thank you for answering.
I did the tests you're asking for, but with no "valuable" result : Whatever the "enable scheduled QAP menu refresh" is checked or unchecked, I get the same result : QAP is not responding when updating the "large" database. And this update takes a quite long time. Maybe when I write 30 seconds, it is a little bit exagerated, but this is at least 15 seconds.
I have since this test checked again the option and set the beeps. And there is never more than 1 second (most often less than 1/2 sec) between two beeps, whatever using the "large" database or the "small" (because newly created) one.
I agree with you that 500K or 1 Mb is very tiny for a sqlite database. So most likely the "long time" spent to update it is not "strictly speaking" time spent to update the database; but more time spent by QAP to gather information, compare it to what is already stored in DB , and then define which update has to be done.
I don't know exactly which algorithm you use for this update, but maybe there is an issue if the database contains already a lot of lines in Usage table ?
Currently, the "large" DB contains 987 lines with CollectType="RecentItems" and 393 lines with CollectType="Menu", while respectively 104 and 14 in the "small" db. -
RE: QAP not responding while updating "Frequent" DB
Jean Lalonde > 2020-08-16 15:46
Bonjour Laurent,
> Maybe when I write 30 seconds, it is a little bit exagerated, but this is at least 15 seconds.
This is not the normal time QAP takes to update the Frequent.DB database. As I wrote earlier, this normally takes 150-200 ms.
> I have since this test checked again the option and set the beeps. And there is never more than 1 second (most often less than 1/2 sec) between two beeps, whatever using the "large" database or the "small" (because newly created) one.
OK, this was for the Scheduled QAP Menu refresh. This confirms that updating the menu takes the normal time. This delay does not depend on the size of the database but on the size of your menu: number of items, submenus, shared menus and Live folders.
> So most likely the "long time" spent to update it is not "strictly speaking" time spent to update the database; but more time spent by QAP to gather information, compare it to what is already stored in DB , and then define which update has to be done. I don't know exactly which algorithm you use for this update, but maybe there is an issue if the database contains already a lot of lines in Usage table?
No. As said earlier, this takes around 1/5 of one second on my systems and on those of most other users, I presume. What is does: scan the Windows "Recent Items" folder, check for new items (those added to the folder after the time of the last QAP collect) and add them to the database. Depending on your update interval, there may be from 0 to a few new items added each time it runs.
How about disk access? On what type of drive is saved your QAP working folder (and the file QAP_Frequent.DB)?
Also, I'll send you by email a tool that will help see what happens when updating the database.
Jean -
RE: QAP not responding while updating "Frequent" DB
LaurentG > 2020-08-17 02:01
Hi Jean,
my disk (for both program and working folder) is a SSD. No chance the issue is in disk access...
Another idea : Could it be due to some (hidden...) parameter of windows that woud make it (windows) sluggish when QAP is asking for new items in recent folder ?
FYI, my recent folders contains (this morning) 243 .lnk files, the oldest being from December '19, plus 2 directories, AutomaticDestinations, and CustomDestinations.
This is at least the view I may have of C:\Users\Laurent\AppData\Roaming\Microsoft\Windows\Recent thanks to a command window.
But if I have the same look using Windows explorer, it shows me "only" 53 entries, none older than August 10th -
RE: QAP not responding while updating "Frequent" DB
drwinchell@gmail.com > 2020-08-17 21:12
I reported a similar issue a while back. By following the instructions, I turned on beep before and after refresh. This was very helpful (nice design!).
Between the 1st beep and the 2nd one is about 15-20 seconds when QAP does not respond. This is very repeatable.
I have the interval set to 300 seconds, but it does it less than that. Around 4:00-4:30 minutes. -
RE: QAP not responding while updating "Frequent" DB
Jean Lalonde > 2020-08-17 21:46
I remember this, it was a few months ago. You said it happened occasionally. How frequent or occasional is it since then? Is the issue still occur if you disable the auto-refresh (see the screen capture above)?
I had email exchange with Laurent. The issue stopped for now. We'll continue investigating this when it reappear. -
RE: QAP not responding while updating "Frequent" DB
drwinchell@gmail.com > 2020-08-18 14:47
(2020-08-17 21:46)Jean Lalonde Wrote: I remember this, it was a few months ago. You said it happened occasionally. How frequent or occasional is it since then? Is the issue still occur if you disable the auto-refresh (see the screen capture above)?
I had email exchange with Laurent. The issue stopped for now. We'll continue investigating this when it reappear.
Once I turned on the Beep before and after refreshing, I notice it happening every time. I hear the first beep and know I can't launch QAP for 15-20 seconds while it is doing it's thing. Once I hear the 2nd beep, it is back to working normally.
The occasionalness was randomly catching it doing it's thing. Before enabling the beeps, I had know idea what it was doing.
Unchecking the option 'Enable Scheduled QAP Menu Refresh' stops the problem from happening.
I also hear the 2 beeps back-to-back when launching Customize. -
RE: QAP not responding while updating "Frequent" DB
LaurentG > 2020-08-19 11:02
I have eventually understood that problem occurs if QAP_Frequent.DB contains references to a distant host (accessed using format \\{HostName}\.... ) if this host is no more accessible for any reason.
I sent a more detailed e-Mail directly to Jean with more explanations : advantage (for me) : I've been able to explain all details in French....
I'm confident that Jean will propose us a solution.
In the meantime, a "workaround" is to remove in the database these entries (eg. using "DB Browser for SQlite" application) -
RE: QAP not responding while updating "Frequent" DB
Jean Lalonde > 2020-08-19 14:09
Yes, thanks to Laurent's research and tests, this took us back to an issue reported more than one year ago about retrieving icons from folders or documents sitting on network drives that are temporarily offline. These icons are normally required to build the "Recent Folders" and "Recent Files" menus. Trying to retrieve them from an offline drive cause a delay of about 20 seconds when refreshing these menus (instead of the normal 0,2 second normally required). During that time, the QAP menu is not available. This delay occurs only one time even if there are multiple files offline. This is why it has the same duration regardless of the number of recent folders or files to be looked up on the offline drive. This is occurring between the OS (Windows) and the external drive after QAP tried to retrieve the icon for a file. QAP has no control on this delay.
However, what QAP can do is to avoid retrieving icons for items in the "Recent Folders" and "Recent Files" menus. Instead, QAP will display default icons for folders and files. To do that , uncheck the option "Retrieve icons..." in this section of the Options:
![[Image: UPEjdFt.png]](https://i.imgur.com/UPEjdFt.png)
This solved the case for Laurent. Dale, could you check if it does the same for you.
Jean -
RE: QAP not responding while updating "Frequent" DB
drwinchell@gmail.com > 2020-08-19 16:41
I disabled 'Retrieve icons when refreshing Frequent Folders and Frequent Files' and it has not made a difference. None of my files are offline so I don't think this applies in my case.
I continue to hear the 1st beep, lose response to QAP and after the 2nd beep, it resumes normal operation.
After re-enabling the Retrieve icons options, I started receiving an error message in line 3812 of 'Menu name too long'. I had to exit the problem and reload it to resolve this.
I also noticed that I am over the default 'Maximum number of items in Live folders'. Could this have something to do with it? -
RE: QAP not responding while updating "Frequent" DB
Jean Lalonde > 2020-08-19 16:53
> I also noticed that I am over the default 'Maximum number of items in Live folders'. Could this have something to do with it?
Yes, the more items you have in your "Live folder", the longer it will take to refresh the menu. You could review your Live folders settings and see if there are where you can reduce the number of levels, etc. -
RE: QAP not responding while updating "Frequent" DB
LaurentG > 2020-08-20 01:15
Hi Jean,
just one question : Actually, if the "Retrieve icons when refreshing..." box is UN-checked, Frequent files and folders do not have, as expected, their specific icon, but the default one, but on the other hand, Recent files and folders nevertheless DO HAVE their specific icon.
Why different behaviour for Frequent vs Recent ? -
RE: QAP not responding while updating "Frequent" DB
Jean Lalonde > 2020-08-21 07:32
This is true. Earlier in this thread, I should have said that retrieving the icons for files from the "Frequent Folders" and "Frequent Fi9les" menus ONLY cause this delay.
> Why different behaviour for Frequent vs Recent ?
This is a good question. I took a some time to review my code but could not find the answer easily. This is something I coded more than a year ago and I would have to dig deeper to find the explanation. This is probably related to the fact items in the Frequent menus can be older than those in the Recent menu but I'm not sure of this. I may come back to this when I have more time. Thanks again for you help with this :-) -
RE: QAP not responding while updating "Frequent" DB
Jean Lalonde > 2020-08-29 18:31
An update following email exchanges with Laurent. Thanks to detailed tests from Laurent, we were able to pin point the exact issue: when network assets (folders, documents or applications) with location starting with "\\" found in the Frequent items database but currently offline, QAP tries to retrieve their icon. For the first item, QAP is waiting for Windows' response. If the location is offline, Windows times out after 21 seconds and returns control to QAP that displays the unknown item icon. For remaining frequent items in the menu, for the next 30 seconds (approx), Windows does not wait these 21 seconds and immediate returns an error. This is why the delay is always around 20 seconds, regardless of the number if offline items. Having a better understanding of what is happening, we are now currently testing a solution that will avoid this timeout error.