Workaround for slow RepositoryBrowser on large repositories

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Workaround for slow RepositoryBrowser on large repositories

Schnyder Franz
We use the TortoiseSVN (1.7.11) with various internal repositories (Subversion 1.6.16) and the Repository Browser is very slow, more than 2 minutes to show the initial screen, on larger repositories. I already disabled the RepoBrowserPrefetch and the RepoBrowserShowExternals flags but this did not show the hoped for performance improvement. We also see that a Repository Browser generates quite a CPU load on the server.

I found the fixed Issue 180 "Optimize repo browser for big repositories" and saw that the “incomplete” fetch is only used when fetching the URLs up to the root and not for the rest. So I got the TortoiseSVN code of the 1.7.11 tag and changed the Repository Browser so it always uses the incomplete fetch for all SVN::List (dirent_fields=0x0) calls. But again the performance did not improve much.

In a next attempt I changed the fetch_locks parameter inside SVN::List to always be false. I now the repository browser starts in a 1 to 2 seconds! I reverted the change to use the  incomplete fetch for all SVN::List calls and it still starts within 2 seconds. The only thing I loose is the info in the Lock column.

So it looks like the fetch_locks causes the performance problem. I’m not sure where root cause of the problem lies (Subversion, Server, TortoiseSVN) but my change is a workaround for the problem.

Has anyone ever discovered similar performance problems or heard/read from problems with the fetch_locks flag?

I will try to clean up my code changes and introduce an RepoBrowserFetchLocks flag so TortoiseSVN users with the same problem than I, then could disable the lock fetching.

Is there interest in such a patch?

And is there a chance to integrate it into the official TortoiseSVN version, because for us it would be nice to have this workaround in the official version, so we don’t have to manage our own TortoiseSVN version?

Regards
 Franz

------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3042795

To unsubscribe from this discussion, e-mail: [[hidden email]].
Reply | Threaded
Open this post in threaded view
|

Re: Workaround for slow RepositoryBrowser on large repositories

Oto BREZINA-3
On 2013-01-11 22:03, Schnyder Franz wrote:

> We use the TortoiseSVN (1.7.11) with various internal repositories (Subversion 1.6.16) and the Repository Browser is very slow, more than 2 minutes to show the initial screen, on larger repositories. I already disabled the RepoBrowserPrefetch and the RepoBrowserShowExternals flags but this did not show the hoped for performance improvement. We also see that a Repository Browser generates quite a CPU load on the server.
>
> I found the fixed Issue 180 "Optimize repo browser for big repositories" and saw that the “incomplete” fetch is only used when fetching the URLs up to the root and not for the rest. So I got the TortoiseSVN code of the 1.7.11 tag and changed the Repository Browser so it always uses the incomplete fetch for all SVN::List (dirent_fields=0x0) calls. But again the performance did not improve much.
>
> In a next attempt I changed the fetch_locks parameter inside SVN::List to always be false. I now the repository browser starts in a 1 to 2 seconds! I reverted the change to use the  incomplete fetch for all SVN::List calls and it still starts within 2 seconds. The only thing I loose is the info in the Lock column.
>
> So it looks like the fetch_locks causes the performance problem. I’m not sure where root cause of the problem lies (Subversion, Server, TortoiseSVN) but my change is a workaround for the problem.
>
> Has anyone ever discovered similar performance problems or heard/read from problems with the fetch_locks flag?
>
> I will try to clean up my code changes and introduce an RepoBrowserFetchLocks flag so TortoiseSVN users with the same problem than I, then could disable the lock fetching.
Where in GUI would you set those flag? Or fetch all but lock and fetch
lock on request? - click?
Maybe background fetching would be nice ...
>
> Is there interest in such a patch?
>
> And is there a chance to integrate it into the official TortoiseSVN version, because for us it would be nice to have this workaround in the official version, so we don’t have to manage our own TortoiseSVN version?
Good implemented patch have quite good changes to get into release, but
be aware, that not into 1.7.




--
Oto ot(ik) BREZINA

------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3042805

To unsubscribe from this discussion, e-mail: [[hidden email]].
Reply | Threaded
Open this post in threaded view
|

Re: Workaround for slow RepositoryBrowser on large repositories

Stefan Küng
In reply to this post by Schnyder Franz
On 11.01.2013 22:03, Schnyder Franz wrote:

> We use the TortoiseSVN (1.7.11) with various internal repositories
> (Subversion 1.6.16) and the Repository Browser is very slow, more
> than 2 minutes to show the initial screen, on larger repositories. I
> already disabled the RepoBrowserPrefetch and the
> RepoBrowserShowExternals flags but this did not show the hoped for
> performance improvement. We also see that a Repository Browser
> generates quite a CPU load on the server.
>
> I found the fixed Issue 180 "Optimize repo browser for big
> repositories" and saw that the “incomplete” fetch is only used when
> fetching the URLs up to the root and not for the rest. So I got the
> TortoiseSVN code of the 1.7.11 tag and changed the Repository Browser
> so it always uses the incomplete fetch for all SVN::List
> (dirent_fields=0x0) calls. But again the performance did not improve
> much.
>
> In a next attempt I changed the fetch_locks parameter inside
> SVN::List to always be false. I now the repository browser starts in
> a 1 to 2 seconds! I reverted the change to use the  incomplete fetch
> for all SVN::List calls and it still starts within 2 seconds. The
> only thing I loose is the info in the Lock column.
>
> So it looks like the fetch_locks causes the performance problem. I’m
> not sure where root cause of the problem lies (Subversion, Server,
> TortoiseSVN) but my change is a workaround for the problem.
>
> Has anyone ever discovered similar performance problems or heard/read
> from problems with the fetch_locks flag?

I've now tested with 5 different svn repositories and servers, some with
38'000 folders in its root. Fetching the locks took about half a second
for the biggest ones.

So I guess it is something on your server that's causing this slowdown.
For the repositories I tested, the part that uses up most of the fetch
time are the dirent fields - if all are fetched then it takes much much
longer than if those fields are not fetched.

> I will try to clean up my code changes and introduce an
> RepoBrowserFetchLocks flag so TortoiseSVN users with the same problem
> than I, then could disable the lock fetching.
>
> Is there interest in such a patch?

No need, I'll have that change done maybe today.

> And is there a chance to integrate it into the official TortoiseSVN
> version, because for us it would be nice to have this workaround in
> the official version, so we don’t have to manage our own TortoiseSVN
> version?

At least part of this could go even to the 1.7.x branch.

Stefan

--
        ___
   oo  // \\      "De Chelonian Mobile"
  (_,\/ \_/ \     TortoiseSVN
    \ \_/_\_/>    The coolest interface to (Sub)version control
    /_/   \_\     http://tortoisesvn.net

------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3042966

To unsubscribe from this discussion, e-mail: [[hidden email]].
Reply | Threaded
Open this post in threaded view
|

RE: Re: Workaround for slow RepositoryBrowser on large repositories

Johann Moro
> > Is there interest in such a patch?
>
> No need, I'll have that change done maybe today.

Is anything being done about this and if so in what version when can we expect this.

We have been suffering horrible performance from the Repo Browser since we upgraded to TortoiseSVN 1.7 so I'd be very keen to give this a try.

Alternatively if this is not going to make it any time soon, can you guys share the code that needs to be changed so that we can compile our own local copy ?

Thanks

------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3050609

To unsubscribe from this discussion, e-mail: [[hidden email]].
Reply | Threaded
Open this post in threaded view
|

Re: Workaround for slow RepositoryBrowser on large repositories

Lorenz-2
Thorgal wrote:

>> > Is there interest in such a patch?
>>
>> No need, I'll have that change done maybe today.
>
>Is anything being done about this and if so in what version when can we expect this.
>
>We have been suffering horrible performance from the Repo Browser since we upgraded to TortoiseSVN 1.7 so I'd be very keen to give this a try.
>
>Alternatively if this is not going to make it any time soon, can you guys share the code that needs to be changed so that we can compile our own local copy ?

you could try a night build from the 1.7.x branch

        http://nightlybuilds.tortoisesvn.net/1.7.x/

 or from trunk

        http://nightlybuilds.tortoisesvn.net/latest/
--

Lorenz

------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3050618

To unsubscribe from this discussion, e-mail: [[hidden email]].
Reply | Threaded
Open this post in threaded view
|

RE: Re: Workaround for slow RepositoryBrowser on large repositories

Johann Moro
> you could try a night build from the 1.7.x branch
>
> http://nightlybuilds.tortoisesvn.net/1.7.x/
>
>  or from trunk
>
> http://nightlybuilds.tortoisesvn.net/latest/

Thanks for this. I did give it a try but unfortunately I couldn't see this new option in the Advanced Settings.
I then waited for a new official version of TSVN to be released (that happened yesterday) and I still do not see this new option (RepoBrowserShowLocks) in the list for me to enable.

As far as I can see that change has been done though:
https://code.google.com/p/tortoisesvn/source/diff?spec=svn23979&r=23817&format=side&path=/branches/1.7.x/src/TortoiseProc/RepositoryBrowser.cpp&old_path=/branches/1.7.x/src/TortoiseProc/RepositoryBrowser.cpp&old=23468

So is there something I need to do to have this option appear in my Advanced Settings form ?

Thanks

------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3052620

To unsubscribe from this discussion, e-mail: [[hidden email]].