[ros-diffs] [hyperion] 33703: added nls added nls/3rdparty added nls/3rdparty/icu

[Timo Kreuzer]

I hope everyone is clear in his mind about the fact that this makes up 
20 % of the complete reactos source or in other words increases the size 
of the code by 25% ...
> [ros-diffs] [hyperion] 33703: added nls added nls/3rdparty added nls/3rdparty/icu We
> officially welcome IBM's excellent ICU4C library for Unicode support to
> our humble source tree. May our marriage be long, happy and fertile.

[Gregor Brunmar]

Increase of code size by 25% = increased build time by 25%? In that case 
I will have a build time of an hour next time I sync :(

-G

Timo Kreuzer wrote:
> I hope everyone is clear in his mind about the fact that this makes up 
> 20 % of the complete reactos source or in other words increases the size 
> of the code by 25% ...
> > [ros-diffs] [hyperion] 33703: added nls added nls/3rdparty added nls/3rdparty/icu We
> > officially welcome IBM's excellent ICU4C library for Unicode support to
> > our humble source tree. May our marriage be long, happy and fertile.
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

[Steven Edwards]

2008/5/26 Gregor Brunmar [e-mail]:
> Increase of code size by 25% = increased build time by 25%? In that case I
> will have a build time of an hour next time I sync :(

actually I am afraid it will be worse than that. G++ so freaking slow
compiling C++ code it might even add more time than you expect. This
is why we we kept ICU as soft dependency in Wine when we used to use
it for BiDi support.
-- 
Steven Edwards

"There is one thing stronger than all the armies in the world, and
that is an idea whose time has come." - Victor Hugo

[Alex Ionescu]

Thanks for talking out of your asses, as always.

Does it look like the entire source is built? Why don't you look at
icu4ros and see what little files are actually compiled? This is just
a vendor import, as per our rules.

Build times? Some numbers:

Mesa32 (whose source is 50% of our source tree): 4 minutes, 53 seconds
Explorer (G++ source): 2 minutes, 26 seconds
Normaliz+ICU4ROS: 37 seconds.

Go complain somewhere else.

---------- Forwarded message ----------
From: Steven Edwards [e-mail]
Date: Tue, May 27, 2008 at 1:25 AM
Subject: Re: [ros-dev] [ros-diffs] [hyperion] 33703: added nls added
nls/3rdparty added nls/3rdparty/icu
To: ReactOS Development List [e-mail]

2008/5/26 Gregor Brunmar [e-mail]:
> Increase of code size by 25% = increased build time by 25%? In that case I
> will have a build time of an hour next time I sync :(

actually I am afraid it will be worse than that. G++ so freaking slow
compiling C++ code it might even add more time than you expect. This
is why we we kept ICU as soft dependency in Wine when we used to use
it for BiDi support.

--
Steven Edwards

"There is one thing stronger than all the armies in the world, and
that is an idea whose time has come." - Victor Hugo
_______________________________________________
Ros-dev mailing list
[e-mail]
http://www.reactos.org/mailman/listinfo/ros-dev
-- 
Best regards,
Alex Ionescu

[Steven Edwards]

On Mon, May 26, 2008 at 9:10 PM, Alex Ionescu [e-mail] wrote:
> Thanks for talking out of your asses, as always.

Geeze, touchie, notice I said "afraid it will be" as in future tense.
I haven't looked at it yet.
> Does it look like the entire source is built? Why don't you look at
> icu4ros and see what little files are actually compiled? This is just
> a vendor import, as per our rules.
>
> Build times? Some numbers:
>
> Mesa32 (whose source is 50% of our source tree): 4 minutes, 53 seconds
> Explorer (G++ source): 2 minutes, 26 seconds
> Normaliz+ICU4ROS: 37 seconds.
>
> Go complain somewhere else.

My complaint it more geared to g++ in anycase. Fortunately explorer
should be shorter once we have explorer-new and maybe we can tweak the
Mesa build times. The last time I looked we didn't really optimize the
build for it very well but thats been ages ago.
-- 
Steven Edwards

"There is one thing stronger than all the armies in the world, and
that is an idea whose time has come." - Victor Hugo

[Timo Kreuzer]

I don't know what sources you have, Alex, but in my tree (including .svn):
Mesa32 = 41,8 MB = ~8 % of 500 MB
ICU: 98,2 MB = ~20% of 500 MB

If it's only a vendor import and we are only using the files that are in 
the current rbuild file, do we really need to put all of the code from 
the vendor tree in our main tree? The files that are in the rbuild file 
+ headers = ~800 KB... Is someone going to clean this up or do our rules 
say that all the code must stay in the main tree?

Timo

Alex Ionescu wrote:
> Thanks for talking out of your asses, as always.
>
> Does it look like the entire source is built? Why don't you look at
> icu4ros and see what little files are actually compiled? This is just
> a vendor import, as per our rules.
>
> Build times? Some numbers:
>
> Mesa32 (whose source is 50% of our source tree): 4 minutes, 53 seconds
> Explorer (G++ source): 2 minutes, 26 seconds
> Normaliz+ICU4ROS: 37 seconds.
>
> Go complain somewhere else.

[KJK::Hyperion]

Just because Timo asked so nicely:

1) It will only get _bigger_, never smaller. ICU has a lot of stuff but 
by no means all data we need. Notably absent is any kind of support for 
complex input (i.e. IMEs)
2) It will never, ever, ever go away. Don't count on it. I plan to make 
it impossible to ever remove our dependency from ICU once it's put in. 
If it's good enough for embedded devices (it is), it will have to be 
good enough for us, too. If not, we will make it
3) It's mostly data, not code. Data that cannot be found anywhere else, 
at least not in the form ICU puts it. Data that the way overstretched 
ReactOS team cannot afford to maintain. Yes, we need this. Nobody else 
is going to do it. Postponing the issue will gain us nothing but grief
4) Not to mention the algorithms nobody is going to be clever enough to 
implement. Algorithms that are part of the Unicode specification. And 
the "useless" code (samples, extra tools) is a rather trivial part of 
the whole
5) To us, ICU represents a low-maintenance route to implementing almost 
all of the Win32 NLS support, forever, maintained and kept current by a 
reliable third party (IBM). Most, if not all NLS APIs will consist of 
thin wrappers around ICU functions. The module I committed is but a 
sample of how straightforward it is to implement Unicode and I18n APIs 
with ICU - eventually, a large part of kernel32.dll is going to be a 
statically linked ICU (what's currently compiled as "icu4ros"). Other 
DLLs will get the "ICU treatment" too, but kernel32.dll will be the 
major beneficiary
6) ICU is extremely clean C++ code without any "surprises": no RTTI, nor 
exceptions, nor overuse of templates - it doesn't even include PSDK 
headers! I expect it will always compile very fast. Also, it implies no 
threading model, relying on a simple double-checked lock algorithm for 
initialization, and keeping all global data read-only, so it can be used 
anywhere safely
7) The only open question about it is how to handle the datafiles: 
single large file (all of 11 MB, not even _that_ big for a file that 
contains _all_ unicode character properties and names, _all_ character 
sets/codepages, _all_ locales, calendars, timezones, etc.), or several 
ones that can be installed separatedly (but our installer is nowhere 
near up to the task yet). Also, until we have a better rbuild, I'll have 
to build the data file(s) "offline" and commit the binary output to the 
repository. If repository size is such an issue, tell me whether you'd 
prefer 45 MB of text or 11 MB of binary data - to me, it doesn't matter much

Considered that I won't officially add anything from "nls" to the build 
until I have fully working code that passes regression tests, and that 
this could take time, would you rather lend me a hand in accelerating 
the process, or would you rather I do my development in a branch?

[KJK::Hyperion]

KJK::Hyperion ha scritto:
> If repository size is such an issue, tell me whether you'd 
> prefer 45 MB of text or 11 MB of binary data - to me, it doesn't matter much

Naturally, this doesn't make sense. I'll have to commit the binary data 
file anyway, since we can't build it with rbuild. Once I do, we can 
remove icu4ros\icu\source\data, IF size is such a big issue. Personally, 
I'm reluctant to do it because it means tedious work later, undeleting 
stuff as soon as I discover I need it. It makes future vendor drops 
harder, too

[Aleksey Bragin]

Size is not that big issue, provided the data files won't be updated  
everyday (thus giving a need to redownload 11Mb for our precious  
dialup and GPRS users).

Also, before it's gone too far, let's be consistent and move icu to  
lib/3rdparty, normalize to dlls/win32, etc.
As for the branch - I would prefer trunk directly.

WBR,
Aleksey Bragin.

On May 31, 2008, at 8:05 AM, KJK::Hyperion wrote:
> KJK::Hyperion ha scritto:
> > If repository size is such an issue, tell me whether you'd
> > prefer 45 MB of text or 11 MB of binary data - to me, it doesn't  
> > matter much
>
> Naturally, this doesn't make sense. I'll have to commit the binary  
> data
> file anyway, since we can't build it with rbuild. Once I do, we can
> remove icu4ros\icu\source\data, IF size is such a big issue.  
> Personally,
> I'm reluctant to do it because it means tedious work later, undeleting
> stuff as soon as I discover I need it. It makes future vendor drops
> harder, too

[KJK::Hyperion]

Aleksey Bragin ha scritto:
> Size is not that big issue, provided the data files won't be updated 
> everyday (thus giving a need to redownload 11Mb for our precious dialup 
> and GPRS users).

I'll keep that in mind. Also, I will probably commit the binary data
files as compilable C source - it's better for me, and it presumably
will be better for people with slow connections too. I don't expect it
will be updated often, either. The data should be binary-compatible
across minor releases, so we could even track minor releases without
necessarily rebuilding all data
> Also, before it's gone too far, let's be consistent and move icu to 
> lib/3rdparty, normalize to dlls/win32, etc.

I want to keep it separate until 0.3.5 is released, so it will be easier
for you to exclude it from the release branch (I don't think I can make
something usable for the release)