Rescanning the bindings - I'm about to cry

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Rescanning the bindings - I'm about to cry

Tommaso Pecorella
I'm about to cry and give up...

Linux box, with Ubuntu 18.04.

I downloaded bake from git, and configured with:
./bake.py configure -e ns-3-dev -e pygccxml-1.9.1
./bake.py download
./bake.py build (fails, obviously, because it wants new bindings).

Now I should rebuild the bindings.

castxml version 0.3.20200427-g8625aed (downloaded from the official repos, as the one downloaded by bake is too old).
pygccxml-1.9.1
pybindgen 0.21.0 (downloaded by bake)
Python 3.6.9 (using a venv)

Trying to rescan an "old" point where I know that things should have been right (147b484a155fc4d46e049f227667093b07575ceb).

Scanning fails.

What am I doing wrong ?

The problem is also in the "error" messages. There are so many logs that's it is almost impossible to understand what's an informative message, what's a warning, and what is the actual error.

Cheers,

T.


--------------------------------------------------------------

In protocol design, perfection has been reached not when there
is nothing left to add, but when there is nothing left to take
away.
-- R. Callon, RFC 1925

--------------------------------------------------------------

Tommaso Pecorella - Ph.D.

Assistant professor
Dpt. Ingegneria dell'Informazione
Università di Firenze

CNIT - Università di Firenze Unit

via di S. Marta 3
50139, Firenze
ITALY

email: [hidden email]
       [hidden email]

phone : +39-055-2758540
mobile: +39-320-4379803
fax   : +39-055-2758570






Reply | Threaded
Open this post in threaded view
|

Re: Rescanning the bindings - I'm about to cry

Tom Henderson-2
On 6/2/20 5:57 AM, Tommaso Pecorella wrote:
> I'm about to cry and give up...
>
> Linux box, with Ubuntu 18.04.
>
> I downloaded bake from git, and configured with:
> ./bake.py configure -e ns-3-dev -e pygccxml-1.9.1

Your pygccxml is too old; you need to use the dev version.  If you use
bake.py to configure the 'ns-3-allinone' target, it should work.

If the instructions somewhere say to use pygccxml-1.9.1, we should fix that.

- Tom


Reply | Threaded
Open this post in threaded view
|

Re: Rescanning the bindings - I'm about to cry

Tommaso Pecorella
Funny enough, pygccxml "dev" still claims to be 1.9.1.

I downloaded it manually and now it works. Tests going on.
I'll let you know how they go.


T.


> On 2 Jun 2020, at 15:47, Tom Henderson <[hidden email]> wrote:
>
> On 6/2/20 5:57 AM, Tommaso Pecorella wrote:
>> I'm about to cry and give up...
>>
>> Linux box, with Ubuntu 18.04.
>>
>> I downloaded bake from git, and configured with:
>> ./bake.py configure -e ns-3-dev -e pygccxml-1.9.1
>
> Your pygccxml is too old; you need to use the dev version.  If you use bake.py to configure the 'ns-3-allinone' target, it should work.
>
> If the instructions somewhere say to use pygccxml-1.9.1, we should fix that.
>
> - Tom
>
>

--------------------------------------------------------------

Thinking evolution:
 "To be is to do" - Socrates
 "To do is to be" - Sartre
 "Do Be Do Be Do" - Sinatra
 "Scooby Dooby Do" - Scooby Do
 "Yaba Daba Doo!" - Fred Flintstone

--------------------------------------------------------------

Tommaso Pecorella - Ph.D.

Assistant professor
Dpt. Ingegneria dell'Informazione
Università di Firenze

CNIT - Università di Firenze Unit

via di S. Marta 3
50139, Firenze
ITALY

email: [hidden email]
       [hidden email]

phone : +39-055-2758540
mobile: +39-320-4379803
fax   : +39-055-2758570








Reply | Threaded
Open this post in threaded view
|

Re: Rescanning the bindings - I'm about to cry

Natale Patriciello
On 02/06/20 at 04:36pm, Tommaso Pecorella wrote:
> Funny enough, pygccxml "dev" still claims to be 1.9.1.
>
> I downloaded it manually and now it works. Tests going on.
> I'll let you know how they go.


There's a gitlab CI script that can compile them for you... Just setup
it within your development repo, and create a manual schedule on the
branch you need. No need to cry :)

Nat

Reply | Threaded
Open this post in threaded view
|

Re: Rescanning the bindings - I'm about to cry

Natale Patriciello
On 03/06/20 at 12:06am, Barnes, Peter D. wrote:
> As far as I can tell this only works if you can already build the
> bindings locally.  If you can’t, do you need to wait for !293 to be
> merged?  So the GitLab CI runs it for you?

Yes, as the generation process is the same as you would do locally. For
instance, without fixing #215 (about the variadic template), the CI/CD
machine would not be able to generate the bindings.

Even without !215 (ci/cd script MR) merged, you can setup a branch with such script, and
then create a schedule under your development repo, telling against what
branch (and what module) you need to generate the bindings. Then, after
generation, download the artifact, check, and apply to master. Once !215 is
merged, the first step of copying the commit will not be required.

The schedule creation requires two variables:

- PYBINDGEN set to "True" (no quotes)
- MODULE set to the module you want to compile (passed to the command
  line --apiscan=)


Nat


Reply | Threaded
Open this post in threaded view
|

Re: Rescanning the bindings - I'm about to cry

Tommaso Pecorella
Good to know !

Fits some use-cases, and not some others.
The problem is... speed.

The compilation time on gitlab is far longer than on a "normal" computer - and I fear that generating all the bindings will take like half a day or more.
Anyway, I'll give it a try, just to check the speed.

For the records, I was finally able to setup my Linux box to rescan the bindings. What took me on the (definitely) wrong direction was... bake.
I'd like to thanks Zoraze for pointing me to the right direction.
Installing CastXML, pygccxml, and pybindgen from their git repos did the trick.

As a side note, pybindgen "version" can trick you badly, as the development version reports itself as "1.9.1" - which is a also a tag number for an outdated (and not working) version.

Cheers,

T.



> On 3 Jun 2020, at 09:26, Natale Patriciello <[hidden email]> wrote:
>
> On 03/06/20 at 12:06am, Barnes, Peter D. wrote:
>> As far as I can tell this only works if you can already build the
>> bindings locally.  If you can’t, do you need to wait for !293 to be
>> merged?  So the GitLab CI runs it for you?
>
> Yes, as the generation process is the same as you would do locally. For
> instance, without fixing #215 (about the variadic template), the CI/CD
> machine would not be able to generate the bindings.
>
> Even without !215 (ci/cd script MR) merged, you can setup a branch with such script, and
> then create a schedule under your development repo, telling against what
> branch (and what module) you need to generate the bindings. Then, after
> generation, download the artifact, check, and apply to master. Once !215 is
> merged, the first step of copying the commit will not be required.
>
> The schedule creation requires two variables:
>
> - PYBINDGEN set to "True" (no quotes)
> - MODULE set to the module you want to compile (passed to the command
>  line --apiscan=)
>
>
> Nat
>
>

--------------------------------------------------------------

Who says nothing is impossible?
  Some people do it every day!
-- Alfred E. Neuman

--------------------------------------------------------------

Tommaso Pecorella - Ph.D.

Assistant professor
Dpt. Ingegneria dell'Informazione
Università di Firenze

CNIT - Università di Firenze Unit

via di S. Marta 3
50139, Firenze
ITALY

email: [hidden email]
       [hidden email]

phone : +39-055-2758540
mobile: +39-320-4379803
fax   : +39-055-2758570








Reply | Threaded
Open this post in threaded view
|

Re: Rescanning the bindings - I'm about to cry

Natale Patriciello
On 03/06/20 at 10:33am, Tommaso Pecorella wrote:
> Good to know !
>
> Fits some use-cases, and not some others.
> The problem is... speed.
>
> The compilation time on gitlab is far longer than on a "normal" computer - and I fear that generating all the bindings will take like half a day or more.
> Anyway, I'll give it a try, just to check the speed.
[cut]


The speed is limited by the number of processors that are in the VM that
execute the job (probably, one). Often, you need to generate the
bindings only for one module; LTE and WiFi are biggest one (and should
stay inside the 3h mark, or the job will fail).

A full-generation can be made faster by creating as many schedules as
modules, and running them in parallel.

For me, is something that I reserve for the nights: a click, and then go
to bed, and the next morning I can see the result.


Nat