proposed changes to ns-2 RED code

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

proposed changes to ns-2 RED code

Tom Henderson-2
If you are using ns-2 and RED queues, please help us to evaluate the
following proposed change.

Mohit Tahiliani has been working on Adaptive RED in ns-2, and has
patched a few issues, including:

-   use of ARED in wireless networks leads to floating point exception
-   the default value of Queue/RED set q_weight_ -1 is incorrect
-   Queue/RED set adaptive_ 0: this must be set to 1, otherwise "max_p"
parameter never adapts.

While the default values of some parameters (such as thresh_,
maxthresh_, q_weight_) were changed in 2001 to make ARED as the default
RED mechanism in ns-2, those of others parameters were
left unchanged. The resulting code defaults to something that is neither
RED nor ARED; this patch will fix the default to ARED.

The proposed patch is in a tracker issue here:

http://sourceforge.net/p/nsnam/patches/25/

I'm testing release candidates for ns-2.36, which are described here:

http://nsnam.isi.edu/nsnam/index.php/Roadmap

Mohit's patch is _not_ part of the first release candidate.  If we move
forward with it, it will be merged as part of a later release candidate.
  So to test it yourself, I recommend to download the release candidate
and apply the patch there.

I've been through a couple of review cycles with Mohit on this patch.
We'll use lazy consensus to try to decide on its inclusion.  Unless we
hear from the community that these changes should be reconsidered (let's
set a date, such as by January 10), I plan to work with Mohit to
evaluate the ns-2 validate trace changes and update the traces, and
commit this to ns-2 prior to the ns-2.36 release.

Of course, even if you support this, it would be nice to hear
positive feedback if you read over this patch, test it, and like what
you see.

Thanks,
Tom
Reply | Threaded
Open this post in threaded view
|

Re: proposed changes to ns-2 RED code

Dave Taht
so you are saying the RED research of the last 12 years has all been
invalid? That is a lot of papers to ashcan?

On Sun, Dec 21, 2014 at 12:07 PM, Tom Henderson <[hidden email]> wrote:

> If you are using ns-2 and RED queues, please help us to evaluate the
> following proposed change.
>
> Mohit Tahiliani has been working on Adaptive RED in ns-2, and has
> patched a few issues, including:
>
> -   use of ARED in wireless networks leads to floating point exception
> -   the default value of Queue/RED set q_weight_ -1 is incorrect
> -   Queue/RED set adaptive_ 0: this must be set to 1, otherwise "max_p"
> parameter never adapts.
>
> While the default values of some parameters (such as thresh_,
> maxthresh_, q_weight_) were changed in 2001 to make ARED as the default
> RED mechanism in ns-2, those of others parameters were
> left unchanged. The resulting code defaults to something that is neither
> RED nor ARED; this patch will fix the default to ARED.
>
> The proposed patch is in a tracker issue here:
>
> http://sourceforge.net/p/nsnam/patches/25/
>
> I'm testing release candidates for ns-2.36, which are described here:
>
> http://nsnam.isi.edu/nsnam/index.php/Roadmap
>
> Mohit's patch is _not_ part of the first release candidate.  If we move
> forward with it, it will be merged as part of a later release candidate.
>  So to test it yourself, I recommend to download the release candidate
> and apply the patch there.
>
> I've been through a couple of review cycles with Mohit on this patch.
> We'll use lazy consensus to try to decide on its inclusion.  Unless we
> hear from the community that these changes should be reconsidered (let's
> set a date, such as by January 10), I plan to work with Mohit to
> evaluate the ns-2 validate trace changes and update the traces, and
> commit this to ns-2 prior to the ns-2.36 release.
>
> Of course, even if you support this, it would be nice to hear
> positive feedback if you read over this patch, test it, and like what
> you see.
>
> Thanks,
> Tom



--
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks

Reply | Threaded
Open this post in threaded view
|

Re: proposed changes to ns-2 RED code

Tom Henderson-2
On 12/21/2014 12:31 PM, Dave Taht wrote:
> so you are saying the RED research of the last 12 years has all been
> invalid? That is a lot of papers to ashcan?

I don't know how people have been using the RED models over the years.
It may be that researchers have been tuning the parameters to use
non-default values.  Anyway, those who have been using RED in ns-2 ought
to have a look and see if they agree with Mohit's suggestions.

I posted the summary on Mohit's behalf, so maybe he can elaborate more
on the changes that he is proposing.

- Tom
Reply | Threaded
Open this post in threaded view
|

Re: proposed changes to ns-2 RED code

Mohit P. Tahiliani
In reply to this post by Tom Henderson-2
Hi all,

Thanks to Tom Sir for posting about the RED patch.

Dave Sir - Following are the details of the proposed changes:

There are two aspects of the patch:

1. Simulating Adaptive RED (ARED) in wireless networks in ns-2 gives
"floating point exception".

The proposed patch aims to solve this problem. Thanks to Tom Sir again for
reviewing it several times.

2. Combination of the default values of "thresh_, maxthresh_, q_weight_,
and adaptive_" in ns-2 neither make Original RED nor ARED as a default AQM.

Regarding RED's research since last 12 years -

I believe *ARED* has not yet been simulated / studied thoroughly in
*wireless networks using ns-2*. Hence, the problem of "floating point
exception" which I mentioned in point 1 above *may* have gone un-noticed
(Please correct me if I am mistaken).

Like Tom Sir mentioned, I too guess that most of the papers that simulate
RED use their own values for "thresh_, maxthresh_, q_weight_ and max_p"
rather than using the default values of ns-2.

For example, DCTCP authors use their own values for setting RED parameters
while testing DCTCP's performance in the presence of RED. Similar is the
case with LTTCP and a few other papers that I have read.

I request you to give your suggestions on the same and guide me, if I am
mistaken at any point.

Best Regards,
Mohit P. Tahiliani

On Mon, Dec 22, 2014 at 6:02 AM, Tom Henderson <[hidden email]> wrote:

> On 12/21/2014 12:31 PM, Dave Taht wrote:
>
>> so you are saying the RED research of the last 12 years has all been
>> invalid? That is a lot of papers to ashcan?
>>
>
> I don't know how people have been using the RED models over the years. It
> may be that researchers have been tuning the parameters to use non-default
> values.  Anyway, those who have been using RED in ns-2 ought to have a look
> and see if they agree with Mohit's suggestions.
>
> I posted the summary on Mohit's behalf, so maybe he can elaborate more on
> the changes that he is proposing.
>
> - Tom
>
Reply | Threaded
Open this post in threaded view
|

Re: proposed changes to ns-2 RED code

Abrar Khan
Hi all,

I am curious to know if there are any guidelines/steps to (i) contribute
code to ns2 and (ii) for validation tests (other than "validate" in ns2)?

I have came across the following but needs more guidance please.

http://www.isi.edu/nsnam/ns/ns-contribute-howto.html

http://www.isi.edu/nsnam/ns/ns-tests.html

I am particularly interesting in finding documents like this one (ns3)

http://www.nsnam.org/docs/release/3.10/testing/testing.pdf

I am very thankful to you for reading my email. I welcome your comments and
apologise if I couldn't put my question in a professional way.

Many thanks




*Abrar Khan*


On 22 December 2014 at 09:08, Mohit P. Tahiliani <[hidden email]>
wrote:

> Hi all,
>
> Thanks to Tom Sir for posting about the RED patch.
>
> Dave Sir - Following are the details of the proposed changes:
>
> There are two aspects of the patch:
>
> 1. Simulating Adaptive RED (ARED) in wireless networks in ns-2 gives
> "floating point exception".
>
> The proposed patch aims to solve this problem. Thanks to Tom Sir again for
> reviewing it several times.
>
> 2. Combination of the default values of "thresh_, maxthresh_, q_weight_,
> and adaptive_" in ns-2 neither make Original RED nor ARED as a default AQM.
>
> Regarding RED's research since last 12 years -
>
> I believe *ARED* has not yet been simulated / studied thoroughly in
> *wireless networks using ns-2*. Hence, the problem of "floating point
> exception" which I mentioned in point 1 above *may* have gone un-noticed
> (Please correct me if I am mistaken).
>
> Like Tom Sir mentioned, I too guess that most of the papers that simulate
> RED use their own values for "thresh_, maxthresh_, q_weight_ and max_p"
> rather than using the default values of ns-2.
>
> For example, DCTCP authors use their own values for setting RED parameters
> while testing DCTCP's performance in the presence of RED. Similar is the
> case with LTTCP and a few other papers that I have read.
>
> I request you to give your suggestions on the same and guide me, if I am
> mistaken at any point.
>
> Best Regards,
> Mohit P. Tahiliani
>
> On Mon, Dec 22, 2014 at 6:02 AM, Tom Henderson <[hidden email]> wrote:
>
> > On 12/21/2014 12:31 PM, Dave Taht wrote:
> >
> >> so you are saying the RED research of the last 12 years has all been
> >> invalid? That is a lot of papers to ashcan?
> >>
> >
> > I don't know how people have been using the RED models over the years. It
> > may be that researchers have been tuning the parameters to use
> non-default
> > values.  Anyway, those who have been using RED in ns-2 ought to have a
> look
> > and see if they agree with Mohit's suggestions.
> >
> > I posted the summary on Mohit's behalf, so maybe he can elaborate more on
> > the changes that he is proposing.
> >
> > - Tom
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: proposed changes to ns-2 RED code

Dave Taht
In reply to this post by Mohit P. Tahiliani
On Mon, Dec 22, 2014 at 1:08 AM, Mohit P. Tahiliani
<[hidden email]> wrote:

> Hi all,
>
> Thanks to Tom Sir for posting about the RED patch.
>
> Dave Sir - Following are the details of the proposed changes:
>
> There are two aspects of the patch:
>
> 1. Simulating Adaptive RED (ARED) in wireless networks in ns-2 gives
> "floating point exception".
>
> The proposed patch aims to solve this problem. Thanks to Tom Sir again for
> reviewing it several times.
>
> 2. Combination of the default values of "thresh_, maxthresh_, q_weight_, and
> adaptive_" in ns-2 neither make Original RED nor ARED as a default AQM.
>
> Regarding RED's research since last 12 years -
>
> I believe *ARED* has not yet been simulated / studied thoroughly in
> *wireless networks using ns-2*. Hence, the problem of "floating point
> exception" which I mentioned in point 1 above *may* have gone un-noticed
> (Please correct me if I am mistaken).
>
> Like Tom Sir mentioned, I too guess that most of the papers that simulate
> RED use their own values for "thresh_, maxthresh_, q_weight_ and max_p"
> rather than using the default values of ns-2.

Given that so few papers publish their test sources it is impossible to know.

There are many fairly recent ones (look on scholar.google.com for
"bufferbloat"),
that may be possible to check; some are people that I know personally.

I will pursue this over the holidays.

> For example, DCTCP authors use their own values for setting RED parameters
> while testing DCTCP's performance in the presence of RED. Similar is the
> case with LTTCP and a few other papers that I have read.

Those folk were "special", they knew what they were doing.

There are tons of papers using RED as a reference AQM
that I sincerely doubt took anything other than the defaults.

> I request you to give your suggestions on the same and guide me, if I am
> mistaken at any point.

If there was a way to mark every paper published on RED as suspect due
to this reason, on google scholar and the various journals, and make the
academics recheck everything, I would.

In fact, I would like that feature for all kinds of papers...

>
> Best Regards,
> Mohit P. Tahiliani
>
> On Mon, Dec 22, 2014 at 6:02 AM, Tom Henderson <[hidden email]> wrote:
>>
>> On 12/21/2014 12:31 PM, Dave Taht wrote:
>>>
>>> so you are saying the RED research of the last 12 years has all been
>>> invalid? That is a lot of papers to ashcan?
>>
>>
>> I don't know how people have been using the RED models over the years. It
>> may be that researchers have been tuning the parameters to use non-default
>> values.  Anyway, those who have been using RED in ns-2 ought to have a look
>> and see if they agree with Mohit's suggestions.
>>
>> I posted the summary on Mohit's behalf, so maybe he can elaborate more on
>> the changes that he is proposing.
>>
>> - Tom
>
>



--
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks

Reply | Threaded
Open this post in threaded view
|

Re: proposed changes to ns-2 RED code

Natale Patriciello
In reply to this post by Abrar Khan
On 22/12/14 at 04:39pm, Abrar Khan wrote:
> Hi all,
>
> I am curious to know if there are any guidelines/steps to (i) contribute
> code to ns2 and (ii) for validation tests (other than "validate" in ns2)?


In last GLOBECOM conferences, in my track, 4 paper out of 5 used ns2 as
simulation platform (and no one released the sources of their
experiments).

In my opinion, research without reproducibility of results is not
research but lies; anyway, the recent ns2 RED problems have rise up
(again) a question: why ns2? There is ns3. Start porting codes and
feature you need (I have done that with TCP options, so I know what I'm
talking about). IMHO, the release of another ns2 version could end up
with researcher thinking "oh, ns2 is still alive, let's continue to use
it" .. leaving ns3 with 1/5 of the researcher user base. If it is the
trend, I suppose that I should start to learn how to use ns2...

Have a nice day

Nat
Reply | Threaded
Open this post in threaded view
|

Re: proposed changes to ns-2 RED code

Tom Henderson-2
On 12/22/2014 10:12 AM, Natale Patriciello wrote:

> IMHO, the release of another ns2 version could end up
> with researcher thinking "oh, ns2 is still alive, let's continue to use
> it" .. leaving ns3 with 1/5 of the researcher user base. If it is the
> trend, I suppose that I should start to learn how to use ns2...

I recognize that some people may disagree with making a new ns-2
release, and supporting both tools leads to fragmentation.  However,
some researchers are still using ns-2 productively, and not maintaining
it leads to forks and clutter.  The ICCRG has recently rekindled a TCP
evaluation suite that is based on ns-2:
http://trac.tools.ietf.org/group/irtf/trac/wiki/ICCRG_tcpeval
and some folks have invested a lot of effort in the past year or two on
cable and AQM models.

So, I am willing to do some amount of ns-2 maintenance to help those
people who are invested in the toolchain.  However, I'd also like to
show people (especially new users) the benefits of starting new projects
with ns-3, and cutting over ns-2 work to ns-3 over time.

I have discussed these issues with a few people in the past year or two.
  I suggested that we ought to consolidate the documentation for both
projects on nsnam.org, help people with transition from ns-2 to ns-3,
make clear the choices available to new users, and better highlight the
large body of research that has already been published with ns-3.  I've
also been considering whether to integrate the build systems using bake
(allowing ns-2 and nam to be installed alongside of ns-3 and DCE).  I
would have preferred to be further along with these types of things by
now, but I'd like to do more of it in the first half of next year.

- Tom
Reply | Threaded
Open this post in threaded view
|

Re: proposed changes to ns-2 RED code

Tom Henderson-2
In reply to this post by Abrar Khan
On 12/22/2014 08:39 AM, Abrar Khan wrote:

> Hi all,
>
> I am curious to know if there are any guidelines/steps to (i) contribute
> code to ns2 and (ii) for validation tests (other than "validate" in ns2)?
>
> I have came across the following but needs more guidance please.
>
> http://www.isi.edu/nsnam/ns/ns-contribute-howto.html
>
> http://www.isi.edu/nsnam/ns/ns-tests.html
>
> I am particularly interesting in finding documents like this one (ns3)
>
> http://www.nsnam.org/docs/release/3.10/testing/testing.pdf
>
> I am very thankful to you for reading my email. I welcome your comments and
> apologise if I couldn't put my question in a professional way.
>

Abrar,

The ns-3 document that you cite has been incorporated into the ns-3
manual, so those details are now found in the manual, such as at this URL:

http://www.nsnam.org/docs/release/3.21/manual/html/tests.html

As for guidelines/steps to contribute code to ns-2, I have been asking
people to follow the steps outlined in the page that you cite:

http://www.isi.edu/nsnam/ns/ns-contribute-howto.html

and also the wiki page guidance here:

http://nsnam.isi.edu/nsnam/index.php/Contributing

Taking the recent AQM additions to ns-2 as an example, the authors were
interested in getting their models into the main tree, so I worked with
them to help them complete steps (5), (6), and (7) from their original
submissions.

As for guidelines for writing tests, there are not any documents for
ns-2 comparable to the one that you pointed out for ns-3.  The one that
you referenced is probably the best one that I know about.  Usually,
people will clone and modify some existing test suite once they learn
how they work.

There are significant differences in how ns-2 and ns-3 have approached
validation and regression tests.  We took a quite different approach for
ns-3 because the ns-2 approach that relies heavily on trace file
comparison has proven to be very difficult to maintain, and the details
about what exactly has been validated tend to be lost over time.

- Tom
Reply | Threaded
Open this post in threaded view
|

Re: proposed changes to ns-2 RED code

Tom Henderson-2
In reply to this post by Tom Henderson-2
On 12/21/2014 12:07 PM, Tom Henderson wrote:

> If you are using ns-2 and RED queues, please help us to evaluate the
> following proposed change.
>
> Mohit Tahiliani has been working on Adaptive RED in ns-2, and has
> patched a few issues, including:
>
> -   use of ARED in wireless networks leads to floating point exception
> -   the default value of Queue/RED set q_weight_ -1 is incorrect
> -   Queue/RED set adaptive_ 0: this must be set to 1, otherwise "max_p"
> parameter never adapts.
>
> While the default values of some parameters (such as thresh_,
> maxthresh_, q_weight_) were changed in 2001 to make ARED as the default
> RED mechanism in ns-2, those of others parameters were
> left unchanged. The resulting code defaults to something that is neither
> RED nor ARED; this patch will fix the default to ARED.
>
> The proposed patch is in a tracker issue here:
>
> http://sourceforge.net/p/nsnam/patches/25/
>
> I'm testing release candidates for ns-2.36, which are described here:
>
> http://nsnam.isi.edu/nsnam/index.php/Roadmap
>
> Mohit's patch is _not_ part of the first release candidate.  If we move
> forward with it, it will be merged as part of a later release candidate.
>   So to test it yourself, I recommend to download the release candidate
> and apply the patch there.
>
> I've been through a couple of review cycles with Mohit on this patch.
> We'll use lazy consensus to try to decide on its inclusion.  Unless we
> hear from the community that these changes should be reconsidered (let's
> set a date, such as by January 10), I plan to work with Mohit to
> evaluate the ns-2 validate trace changes and update the traces, and
> commit this to ns-2 prior to the ns-2.36 release.
>
> Of course, even if you support this, it would be nice to hear
> positive feedback if you read over this patch, test it, and like what
> you see.
>
> Thanks,
> Tom

Hi all,

I've looked into this further and learned that we may not have a major
problem to fix with RED in ns-2.  It turns out that there is a
distinction between what is the default in ns-2, and is called
"automatic configuration of RED parameters", and what is called
"adaptive RED".  This is discussed in more detail on Sally Floyd's RED
page:

http://www.icir.org/floyd/red.html#parameters

So, the claim that adaptive RED was made the default in ns-2 a while
back is not correct.  This wasn't very clear in the documentation.

Accordingly, I updated the patch in the sourceforge tracker to fix the
segmentation fault caused by mixing adaptive RED with wireless links,
and to update the RED documentation about the ns-2 defaults.  This patch
does not cause any of the existing validation output to change.  I'm
planning to add it to the main tree if there are no other comments.

https://sourceforge.net/p/nsnam/patches/25/

To respond to a ns-2 RED question on another mailing list, I decided to
try to run the existing RED tests against the tests documented in this
1996 report, which was actually conducted using ns-1 (and tests later
ported to ns-2):

http://www.icir.org/floyd/red.html#ns

I was able to pretty closely reproduce most of the figures in that
report by running the existing ns-2 RED test suite, suggesting to me
that the basic RED implementation in ns-2 has been stable for a long time.

I expect to post a new ns-2 release candidate shortly, once I resolve
some validation trace discrepancies that are showing up on a few platforms.

- Tom