Wine libs vendor import

[Aleksey Bragin]

I'm answering to ros-dev so all developers could share their point of  
view (Klemens' original email is attached below).

As I basically outlined in an irc conversation, there are 3 possible  
ways for us to sync with the Wine project:
1) Do manual syncs (the current way), keeping their code in our tree,  
building with our build system
2) Do auto-import of their source code into our tree, plus  
automatically detect changes in their build files and propogate them  
to our .rbuild files
3) Do binary-syncing of the dlls which are free from reactos and wine  
hacks, and do manual syncs for dlls which are forked (like setupapi  
which is better implemented, directx dlls which are partly shared and  
partly implemented differently, etc).

However, it was not possible to easily come to a solution what to do.  
I will give a small overview of all ways:
1) We lag behind, providing outdated, featureless and buggy version  
of dlls. And even more, we depend on dlls bugs, reactos hacks in  
them. Syncing them requires time and constant efforts.
 From the good side we have absolute control over the libraries - we  
can enable/disable debug messages, build them for different  
architectures, optimize for different CPUs etc
2) This could sound like an ideal way, because it keeps advantages of  
1. plus removing disadvantages of having to waste time doing syncs.  
However conversion from Wine's build system makefiles to ReactOS's  
build system .rbuild isn't trivial. Plus developing an auto-syncer  
between Cvs/Git and our SVN doesn't look like an easy task too.
3) Reduces compilation time, however wine dlls are built on another  
server, and just downloaded during build process (or kept in SVN  
server in binary form). It'd be needed to adjust building mechanism  
to be able to produce other platform binaries. Updates will be  
"heavy" (but initial download smaller).


I'm listening to your (all ReactOS and Wine devs) comments, and  
probably I will write answers to Klemens' email later.

WBR,
Aleksey Bragin.



On Jun 12, 2006, at 2:35 PM, Klemens Friedl wrote:

>   Hi Aleksey,
>
>  I spoke with Gé and he explained me the vendor import process in
>  short. As soon as I find some time I will try out it myself and update
>  the Wine libs in our SVN tree, if you don't mind.
>
>  Or do you want to switch to a binary wine files in trunk anyway?
>
>  I will try to update Wine files about every months after the first
>  time, if that is okay. The changes of wine 0.9.x of every two week are
>  minor things so, every months should be fine. sdwards mentioned that
>  timeframe too, as we discussed about that topic about one week ago on
>  #winehackers and #reactos (the first time).
>
>  btw. Gé mentioned there ARE reasons why our Wine libs DO differ from
>  WineHQ's one. Well Gé is not a ros dev anymore but he had not done the
>  vendor import work and maintain all the changes for fun, it's
>  important to have our adjustments in there.
>  Imagine for example all icons would got lost.
>
>  Wine does use Unix specific functions like "unixfs" in several libs,
>  the sound libs will get a lot different in near future too
>  (silverblade is working in this area) and several directx libs. So a
>  quick auto vendor import won't work anyway.
>
>
>  In my opinion, the binary idea is not worth any efford:
>
>  In my opinion, binary files on in a svn dir is a very bad solution.
>  You cannot view the source and commit the changes, you have to
>  download all binary files every time (every 2 weeks), binary files are
>  not portable, etc.
>  The advantages of source code is in addition to the oposite of the
>  facts above that you have to download only the diffs of the file
>  changes (svn client does that for you) and that have everything
>  (source) on your local hdd.
>  Sure the build-time would decrease about 10-15% if we use binary
>  files, but i don't bother about 15% time.
>
>  And please shift the wine lib conversation to ros-dev(a)reactos.org.
>  The irc chat is not suitable for such a discussion where more than 3-4
>  devs are involved. It is not possible to answer to more then one msg
>  the same time. And while you write, several other persons post other
>  msg's.
>  Mailing lists does have a purpose and does make sense here!
>  E-Mail's are written in a better english, no short abbreviations, etc.
>  and the devs does think a bit longer about what the write (no offence
>  here!).
>
>
>  Best regards,
>  Klemens

[James Tabor]

Aleksey Bragin wrote:
>   I'm answering to ros-dev so all developers could
> share their point of  
>  view (Klemens' original email is attached below).
>  
>  As I basically outlined in an irc conversation, there are 3 possible  
>  ways for us to sync with the Wine project:
>  1) Do manual syncs (the current way), keeping their code in our tree,  
>  building with our build system
>  2) Do auto-import of their source code into our tree, plus  
>  automatically detect changes in their build files and propogate them  
>  to our .rbuild files
>  3) Do binary-syncing of the dlls which are free from reactos and wine  
>  hacks, and do manual syncs for dlls which are forked (like setupapi  
>  which is better implemented, directx dlls which are partly shared and  
>  partly implemented differently, etc).
>  
>  However, it was not possible to easily come to a solution what to do.  
>  I will give a small overview of all ways:
>  1) We lag behind, providing outdated, featureless and buggy version  
>  of dlls. And even more, we depend on dlls bugs, reactos hacks in  
>  them. Syncing them requires time and constant efforts. 
I'm doing this right now please bare with me, I have allot of research
to go through and testing. Doing it regardless of project approval.

I'm in gdi32 objects, and user32 menu.c and that includes the related
parts in win32k. Also working in kernel32 lib locate and loading failures.
Wine has this problem too.

>    From the good side we have absolute control over the
> libraries - we  
>  can enable/disable debug messages, build them for different  
>  architectures, optimize for different CPUs etc
>  2) This could sound like an ideal way, because it keeps advantages of  
>  1. plus removing disadvantages of having to waste time doing syncs.  
>  However conversion from Wine's build system makefiles to ReactOS's  
>  build system .rbuild isn't trivial. Plus developing an auto-syncer  
>  between Cvs/Git and our SVN doesn't look like an easy task too.
>  3) Reduces compilation time, however wine dlls are built on another  
>  server, and just downloaded during build process (or kept in SVN  
>  server in binary form). It'd be needed to adjust building mechanism  
>  to be able to produce other platform binaries. Updates will be  
>  "heavy" (but initial download smaller).
>  
>  
>  I'm listening to your (all ReactOS and Wine devs) comments, and  
>  probably I will write answers to Klemens' email later.
>  
>  WBR,
>  Aleksey Bragin.
>  
>  
>  
>  On Jun 12, 2006, at 2:35 PM, Klemens Friedl wrote:
>  
>  
> >  Hi Aleksey,
> >
> > I spoke with Gé and he explained me the vendor import process in
> > short. As soon as I find some time I will try out it myself and update
> > the Wine libs in our SVN tree, if you don't mind.
> >
> > Or do you want to switch to a binary wine files in trunk anyway?
> >
> > I will try to update Wine files about every months after the first
> > time, if that is okay. The changes of wine 0.9.x of every two week are
> > minor things so, every months should be fine. sdwards mentioned that
> > timeframe too, as we discussed about that topic about one week ago on
> > #winehackers and #reactos (the first time).
> >
> > btw. Gé mentioned there ARE reasons why our Wine libs DO differ from
> > WineHQ's one. Well Gé is not a ros dev anymore but he had not done the
> > vendor import work and maintain all the changes for fun, it's
> > important to have our adjustments in there.
> > Imagine for example all icons would got lost.
> >
> > Wine does use Unix specific functions like "unixfs" in several libs,
> > the sound libs will get a lot different in near future too
> > (silverblade is working in this area) and several directx libs. So a
> > quick auto vendor import won't work anyway.
> >
> >
> > In my opinion, the binary idea is not worth any efford:
> >
> > In my opinion, binary files on in a svn dir is a very bad solution.
> > You cannot view the source and commit the changes, you have to
> > download all binary files every time (every 2 weeks), binary files are
> > not portable, etc.
> > The advantages of source code is in addition to the oposite of the
> > facts above that you have to download only the diffs of the file
> > changes (svn client does that for you) and that have everything
> > (source) on your local hdd.
> > Sure the build-time would decrease about 10-15% if we use binary
> > files, but i don't bother about 15% time.
> >
> > And please shift the wine lib conversation to ros-dev(a)reactos.org.
> > The irc chat is not suitable for such a discussion where more than 3-4
> > devs are involved. It is not possible to answer to more then one msg
> > the same time. And while you write, several other persons post other
> > msg's.
> > Mailing lists does have a purpose and does make sense here!
> > E-Mail's are written in a better english, no short abbreviations, etc.
> > and the devs does think a bit longer about what the write (no offence
> > here!).
> >
> >
> > Best regards,
> > Klemens 
>  
>  
>  
>  _______________________________________________
>  Ros-dev mailing list
>  Ros-dev(a)reactos.org
>  http://www.reactos.org/mailman/listinfo/ros-dev
>

[Magnus Olsen]

Here is some other point why we need todo manual sync and no prep build

1. if we want build reactos for another os like PPC or Alpha
   we need keep all source code in the svn and compile it.
   I want start portiong ReactOS to my cellphone motoroal
   it using a ARM cpu with Symbian OS. The problem
   is I never get time setting up the env I need for it.


2. some devs are adding own debug msg in wine dll files
    example me. I never use kdb or gdb to debug ReactOS


3. Headers and strucrt does not match windows sdk/ddk size
    wine often using own struct in the header that is not sdk/ddk
    compatible. and some strucrt have wrong size or wrong position
    of a member .


4. We have own private patcher in almost all wine dll
    That never can go to wine or never accpect.


5. I do not want prep build in SVN, I am using around 4-5GB
    band widh each day in d/l.

6. Using diffent version of gcc and optimze settings that gives diffent
    speed, and u can give out optimze build for a cpu.
    (I am using customze setting in my builds)

Thuse people want using prep dll we have that more or less with rbuild
already
frist time comping ReactOS from 100% clean tree it take around 15-25Mins
on a amd64 3000+, on P3 800Mhz around 2-3 Hours. But comping small patch
without doing clean it take around 0-2mins for each commit. if it is not a
header
change. Direlcy when it is a header change in w32sapi or ndk U should do a
make clean.

options 2 and 3 are bad choice and do not provide any good aspect
it will incress alot of problem. I only mentor some of my view about it
and do not like options 2 and 3 it is to bad disavnegs, and creating a
rbuild files for wine code is not hard at all it take around 2-3 mins
for me. Doing a sync with wine it varius with the time. u need
test each sync even it is autmatoc sync. for alot of wine
dll can break ReactOS for they are not using same
registing path as windows nt, There is also other
point of view why we should do manual sync
it IS only one way. It is manual sync.
But if we adding a ros hack or patch to
wine code. we should mark it out
in wine dll. I have done that
in DirectX dll files I tok
from wine. if we mark
out the code every
one will  know
what is changes
and  not
changes
in wine
code

Night
it is pretty late when I wrote this letter



----- Original Message ----- 
From: "Aleksey Bragin" <aleksey(a)studiocerebral.com>
To: "ReactOS Development List" <ros-dev(a)reactos.org>
Sent: Monday, June 12, 2006 11:20 PM
Subject: [ros-dev] Wine libs vendor import


I'm answering to ros-dev so all developers could share their point of
view (Klemens' original email is attached below).

As I basically outlined in an irc conversation, there are 3 possible
ways for us to sync with the Wine project:
1) Do manual syncs (the current way), keeping their code in our tree,
building with our build system
2) Do auto-import of their source code into our tree, plus
automatically detect changes in their build files and propogate them
to our .rbuild files
3) Do binary-syncing of the dlls which are free from reactos and wine
hacks, and do manual syncs for dlls which are forked (like setupapi
which is better implemented, directx dlls which are partly shared and
partly implemented differently, etc).

However, it was not possible to easily come to a solution what to do.
I will give a small overview of all ways:
1) We lag behind, providing outdated, featureless and buggy version
of dlls. And even more, we depend on dlls bugs, reactos hacks in
them. Syncing them requires time and constant efforts.
 From the good side we have absolute control over the libraries - we
can enable/disable debug messages, build them for different
architectures, optimize for different CPUs etc
2) This could sound like an ideal way, because it keeps advantages of
1. plus removing disadvantages of having to waste time doing syncs.
However conversion from Wine's build system makefiles to ReactOS's
build system .rbuild isn't trivial. Plus developing an auto-syncer
between Cvs/Git and our SVN doesn't look like an easy task too.
3) Reduces compilation time, however wine dlls are built on another
server, and just downloaded during build process (or kept in SVN
server in binary form). It'd be needed to adjust building mechanism
to be able to produce other platform binaries. Updates will be
"heavy" (but initial download smaller).


I'm listening to your (all ReactOS and Wine devs) comments, and
probably I will write answers to Klemens' email later.

WBR,
Aleksey Bragin.



On Jun 12, 2006, at 2:35 PM, Klemens Friedl wrote:

>   Hi Aleksey,
>
>  I spoke with Gé and he explained me the vendor import process in
>  short. As soon as I find some time I will try out it myself and update
>  the Wine libs in our SVN tree, if you don't mind.
>
>  Or do you want to switch to a binary wine files in trunk anyway?
>
>  I will try to update Wine files about every months after the first
>  time, if that is okay. The changes of wine 0.9.x of every two week are
>  minor things so, every months should be fine. sdwards mentioned that
>  timeframe too, as we discussed about that topic about one week ago on
>  #winehackers and #reactos (the first time).
>
>  btw. Gé mentioned there ARE reasons why our Wine libs DO differ from
>  WineHQ's one. Well Gé is not a ros dev anymore but he had not done the
>  vendor import work and maintain all the changes for fun, it's
>  important to have our adjustments in there.
>  Imagine for example all icons would got lost.
>
>  Wine does use Unix specific functions like "unixfs" in several libs,
>  the sound libs will get a lot different in near future too
>  (silverblade is working in this area) and several directx libs. So a
>  quick auto vendor import won't work anyway.
>
>
>  In my opinion, the binary idea is not worth any efford:
>
>  In my opinion, binary files on in a svn dir is a very bad solution.
>  You cannot view the source and commit the changes, you have to
>  download all binary files every time (every 2 weeks), binary files are
>  not portable, etc.
>  The advantages of source code is in addition to the oposite of the
>  facts above that you have to download only the diffs of the file
>  changes (svn client does that for you) and that have everything
>  (source) on your local hdd.
>  Sure the build-time would decrease about 10-15% if we use binary
>  files, but i don't bother about 15% time.
>
>  And please shift the wine lib conversation to ros-dev(a)reactos.org.
>  The irc chat is not suitable for such a discussion where more than 3-4
>  devs are involved. It is not possible to answer to more then one msg
>  the same time. And while you write, several other persons post other
>  msg's.
>  Mailing lists does have a purpose and does make sense here!
>  E-Mail's are written in a better english, no short abbreviations, etc.
>  and the devs does think a bit longer about what the write (no offence
>  here!).
>
>
>  Best regards,
>  Klemens 

_______________________________________________
Ros-dev mailing list
Ros-dev(a)reactos.org
http://www.reactos.org/mailman/listinfo/ros-dev