< Previous by Date	Date Index	Next by Date >
< Previous in Thread	Thread Index

Re: [reSIProcate-users] DNS timeout bug in 1.9.x

From: Herbert Karajan <boost.regex@xxxxxxxxx>
Date: Sun, 2 Mar 2014 21:33:23 +0800

Thanks Scott, I will test it and feedback here.

Best regards,

On Sun, Mar 2, 2014 at 2:35 AM, Scott Godin <sgodin@xxxxxxxxxxxxxxx> wrote:

A fix is checked in. I ended up implementing that we just skip fec0 DNS server entries based on the following advice from this discussion:
http://comments.gmane.org/gmane.network.dns.c-ares/1090

// Windows implements a draft RFC from 2001
// (http://tools.ietf.org/id/draft-ietf-ipngwg-dns-discovery-03.txt) that

// suggests that DNS resolvers try a couple of fixed "site-local" IPv6
// addresses (fec0:0:0:ffff::1, fec0:0:0:ffff::2 and fec0:0:0:ffff::3), which
// would allow the networking stack to find DNS servers without any

// configuration.

// The whole concept of site-local addresses is now considered problematic and
// the use of the fec0::/10 address range has been officially deprecated in RFC

// 3879 (http://www.ietf.org/rfc/rfc3879.txt) so it seems unlikely that this
// way of setting up DNS on a network will become popular in the future.

If we need to support these DNS server addresses in the future we can make more elaborate changes at that time. However given the entire address range has been deprecated, I think it's reasonably unlikely we will need to.

Scott

On Sat, Mar 1, 2014 at 1:09 PM, Scott Godin <sgodin@xxxxxxxxxxxxxxx> wrote:

Looks like the c-ares folks found this problem too, since they use the same API the resip ares variant uses. See:

http://c-ares.haxx.se/mail/c-ares-archive-2013-04/0042.shtml

Scott

On Fri, Feb 28, 2014 at 8:42 PM, Herbert Karajan <boost.regex@xxxxxxxxx> wrote:

Hi Scott, I have found that too, the 1.9.x works fine with Win 7 but not works with Win 8 because the DNS initialization give me the DNS serves in below order:

name server: 222.246.129.80
name server: 59.51.78.210
name server: fec0:0:0:ffff::1

name server: fec0:0:0:ffff::2
name server: fec0:0:0:ffff::3

The IPv4 is in begin, so it get works fine, with Windows 8 the IPv6 DNS server is begin the issue is occur.

On Sat, Mar 1, 2014 at 6:43 AM, Scott Godin <sgodin@xxxxxxxxxxxxxxx> wrote:

So it is what I expected. The 1.9 version is returning IPv6 DNS servers first. Since those don't look like they are likely valid DNS servers (fec0 = link local addresses) the lookup is failing.

1.9 gives you the following DNS Server list:
DNS initialization: found 5 name servers
name server: fec0:0:0:ffff::1
name server: fec0:0:0:ffff::2
name server: fec0:0:0:ffff::3

name server: 222.246.129.80
name server: 59.51.78.210

Older version gives you:
DNS initialization: found 2 name servers
name server: 222.246.129.80

name server: 59.51.78.210

Is anyone else experiencing this issue? Or is this perhaps something related to way the way this machine is configured? I wonder why windows is returning 3 IPv6 DNS servers that are not usable.

Scott

On Fri, Feb 28, 2014 at 1:58 AM, Herbert Karajan <boost.regex@xxxxxxxxx> wrote:

Hi, I have modified my application and make more test, and now the application can output all the logs.

Attached the log files:

1_9.log: this log is generate from the resip 1.9.0 release(the 1.9.1 has same issue), in the log, the DNS query take too long time and get time out.

modify_1_9.log: this log is generate from the modified 1.9.0 release(I'm copy some code from the 1.8.5 ares to replace the 1.9.0 code then got the issue fixed, so I call it modified 1.9.0). This log show the DNS query is very fast.

The modifications are here:

I have modified the "static int init_by_defaults_windows_nameservers_getadaptersaddresses" function in 1.9.x then got the issue fixed, the new code is below(in nit_by_defaults_windows_nameservers_getadaptersaddresses function, it called  "get_physical_address" function):

/*
* V4. Get the physical address of the first NIC whose list of DNS servers contain 'addr'.

* return: ARES_SUCCESS, etc.
*/
static int get_physical_address(char *physicalAddr, int physicalAddrBufSz, int* physAddrLen, struct in_addr addr)

{
#ifdef WIN32
DWORD (WINAPI * GetAdaptersAddressesProc)(ULONG, DWORD, VOID *, IP_ADAPTER_ADDRESSES *, ULONG *);

HANDLE hLib = 0;
DWORD dwRet = ERROR_BUFFER_OVERFLOW;
DWORD dwSize = 0;

IP_ADAPTER_ADDRESSES *pAdapterAddresses = 0;
int rc = ARES_ENOTFOUND;

memset(physicalAddr, '\0', physicalAddrBufSz);
*physAddrLen = 0;

hLib = LoadLibrary(TEXT("iphlpapi.dll"));
if (!hLib)
return ARES_ENOTIMP;

(void*)GetAdaptersAddressesProc = GetProcAddress(hLib, TEXT("GetAdaptersAddresses"));
if(!GetAdaptersAddressesProc)

{
rc = ARES_ENOTIMP;
goto cleanup;
}

// Getting buffer size, expects overflow error
dwRet = (*GetAdaptersAddressesProc)(AF_UNSPEC, 0, NULL, NULL, &dwSize);

assert(dwRet == ERROR_BUFFER_OVERFLOW);
if (dwRet == ERROR_BUFFER_OVERFLOW)
{

pAdapterAddresses = (IP_ADAPTER_ADDRESSES *) LocalAlloc(LMEM_ZEROINIT, dwSize);
if (! pAdapterAddresses)
{

rc = ARES_ENOMEM;
goto cleanup;
}
}

else
{
rc = ARES_ENODATA;
goto cleanup;

}

dwRet = (*GetAdaptersAddressesProc)(AF_UNSPEC, 0, NULL, pAdapterAddresses, &dwSize);

if (dwRet != ERROR_SUCCESS)
{
rc = ARES_ENODATA;
goto cleanup;

}

{
IP_ADAPTER_ADDRESSES * AI = NULL;

for (AI = pAdapterAddresses; AI != NULL; AI = AI->Next)
{
PIP_ADAPTER_DNS_SERVER_ADDRESS dnsServers = AI->FirstDnsServerAddress;

// find 'addr' in adapter's list of dns servers.
for (; dnsServers; dnsServers = dnsServers->Next)

{
if (! dnsServers->Address.lpSockaddr)
continue;

if (dnsServers->Address.lpSockaddr->sa_family == AF_INET)
{

struct sockaddr_in sockAddr = *(struct sockaddr_in*)(dnsServers->Address.lpSockaddr);
if (memcmp(&addr, &sockAddr.sin_addr.s_addr, sizeof(struct in_addr)) == 0)

{
*physAddrLen = AI->PhysicalAddressLength;
if (*physAddrLen > physicalAddrBufSz)

*physAddrLen = physicalAddrBufSz;
memcpy(physicalAddr, &AI->PhysicalAddress[0], *physAddrLen);

rc = ARES_SUCCESS;
goto cleanup;
}
}

}
}
}
rc = ARES_ENOTFOUND;

cleanup:
if (hLib)
FreeLibrary(hLib);

if (pAdapterAddresses)
LocalFree(pAdapterAddresses);
return rc;
#else // WIN32
return ARES_ENOTIMP;
#endif // WIN32
}

#ifdef WIN32
static int init_by_defaults_windows_nameservers_getadaptersaddresses(ares_channel channel)

{
/*
* Way of getting nameservers that should work on all Windows from 98 on.

*/
FIXED_INFO * FixedInfo;
ULONG ulOutBufLen;

DWORD dwRetVal;
IP_ADDR_STRING * pIPAddr;
  HANDLE hLib;

  int num;
  DWORD (WINAPI *GetNetworkParams)(FIXED_INFO*, DWORD*);

  hLib = LoadLibrary(TEXT("iphlpapi.dll"));
  if(!hLib)

  {
  return ARES_ENOTIMP;
  }

  (void*)GetNetworkParams = GetProcAddress(hLib, TEXT("GetNetworkParams"));

  if(!GetNetworkParams)
  {
  FreeLibrary(hLib);

  return ARES_ENOTIMP;
  }
//printf("ARES: figuring out DNS servers\n");

FixedInfo = (FIXED_INFO *) GlobalAlloc( GPTR, sizeof( FIXED_INFO ) );
ulOutBufLen = sizeof( FIXED_INFO );

if( ERROR_BUFFER_OVERFLOW == (*GetNetworkParams)( FixedInfo, &ulOutBufLen ) )
  {

GlobalFree( FixedInfo );
FixedInfo = (FIXED_INFO *)GlobalAlloc( GPTR, ulOutBufLen );
}

if ( dwRetVal = (*GetNetworkParams)( FixedInfo, &ulOutBufLen ) )
{

//printf("ARES: couldn't get network params\n");
GlobalFree( FixedInfo );
      FreeLibrary(hLib);

return ARES_ENODATA;
}
else
{

/**
printf( "Host Name: %s\n", FixedInfo -> HostName );
printf( "Domain Name: %s\n", FixedInfo -> DomainName );

printf( "DNS Servers:\n" );
printf( "\t%s\n", FixedInfo -> DnsServerList.IpAddress.String );

**/

// Count how many nameserver entries we have and allocate memory for them.

num = 0;
pIPAddr = &FixedInfo->DnsServerList;
while ( pIPAddr && strlen(pIPAddr->IpAddress.String) > 0)

{
num++;
pIPAddr = pIPAddr ->Next;
}

if(num>0)
{
channel->servers = malloc( (num) * sizeof(struct server_state));

   if (!channel->servers)
   {
   GlobalFree( FixedInfo );
   FreeLibrary(hLib);

   return ARES_ENOMEM;
   }
memset(channel->servers, '\0', num * sizeof(struct server_state));

   channel->nservers = 0;
pIPAddr = &FixedInfo->DnsServerList;

while ( pIPAddr && strlen(pIPAddr->IpAddress.String) > 0)
   {

struct in_addr addr;
addr.s_addr = inet_addr(pIPAddr->IpAddress.String);
// append unique only

if (find_server(channel->servers, channel->nservers, addr) == -1)
{
// printf( "ARES: %s\n", pIPAddr ->IpAddress.String );

#ifdef USE_IPV6
channel->servers[ channel->nservers ].family = AF_INET;

#endif
channel->servers[channel->nservers].addr = addr;
if ((channel->flags & ARES_FLAG_TRY_NEXT_SERVER_ON_RCODE3))

{
get_physical_address(channel->servers[channel->nservers].physical_addr,
MAX_ADAPTER_ADDRESS_LENGTH,

&channel->servers[channel->nservers].physical_addr_len,
addr);

}
   channel->nservers++;
}

pIPAddr = pIPAddr ->Next;
}
//printf("ARES: got all %d nameservers\n",num);

}
else
{
      /* If no specified servers, try a local named. */

   channel->servers = malloc(sizeof(struct server_state));
   if (!channel->servers)

return ARES_ENOMEM;
memset(channel->servers, '\0', sizeof(struct server_state));

#ifdef USE_IPV6
channel->servers[0].family = AF_INET;

#endif

   channel->servers[0].addr.s_addr = htonl(INADDR_LOOPBACK);

channel->servers[0].default_localhost_server = 1;
   channel->nservers = 1;

}

GlobalFree( FixedInfo );
   FreeLibrary(hLib);

  }
  return ARES_SUCCESS;
}

On Wed, Feb 26, 2014 at 10:19 PM, Scott Godin <sgodin@xxxxxxxxxxxxxxx> wrote:

Interesting. I wonder why we aren't see the following log statements in the logs you attached:

InfoLog(<< "DNS initialization: found " << (*channel)->nservers << " name servers");

for (int i = 0; i < (*channel)->nservers; ++i)
{
#ifdef USE_IPV6
if((*channel)->servers[i].family == AF_INET6)
{
InfoLog(<< " name server: " << DnsUtil::inet_ntop((*channel)->servers[i].addr6));

}
else
#endif
{
InfoLog(<< " name server: " << DnsUtil::inet_ntop((*channel)->servers[i].addr));
}

}

This would help us see the different DNS server lists returned to you using the 2 different API's (1.8 version API and 1.9 version API) to understand what is happening. One difference is that the new API used in 1.9 returns ipv6 dns servers as well. Perhaps it is returning a bad server as the first one in the list and the older API is not. Can you try and understand why you are not seeing above log lines from AresDns::::internalInit by stepping through this code? Once you get those to print - let's compare the DNS server lists return from each version to see what's happening.

Thanks,
Scott

On Wed, Feb 26, 2014 at 2:20 AM, Herbert Karajan <boost.regex@xxxxxxxxx> wrote:

Sorry for my mistake, the details is:

I have modified the "static int init_by_defaults_windows_nameservers_getadaptersaddresses" function in 1.9.x then got the issue fixed, the new code is below(in nit_by_defaults_windows_nameservers_getadaptersaddresses function, it called  "get_physical_address" function):

/*
* V4. Get the physical address of the first NIC whose list of DNS servers contain 'addr'.

* return: ARES_SUCCESS, etc.
*/
static int get_physical_address(char *physicalAddr, int physicalAddrBufSz, int* physAddrLen, struct in_addr addr)

{
#ifdef WIN32
DWORD (WINAPI * GetAdaptersAddressesProc)(ULONG, DWORD, VOID *, IP_ADAPTER_ADDRESSES *, ULONG *);

HANDLE hLib = 0;
DWORD dwRet = ERROR_BUFFER_OVERFLOW;
DWORD dwSize = 0;

IP_ADAPTER_ADDRESSES *pAdapterAddresses = 0;
int rc = ARES_ENOTFOUND;

memset(physicalAddr, '\0', physicalAddrBufSz);
*physAddrLen = 0;

hLib = LoadLibrary(TEXT("iphlpapi.dll"));
if (!hLib)
return ARES_ENOTIMP;

(void*)GetAdaptersAddressesProc = GetProcAddress(hLib, TEXT("GetAdaptersAddresses"));
if(!GetAdaptersAddressesProc)

{
rc = ARES_ENOTIMP;
goto cleanup;
}

// Getting buffer size, expects overflow error
dwRet = (*GetAdaptersAddressesProc)(AF_UNSPEC, 0, NULL, NULL, &dwSize);

assert(dwRet == ERROR_BUFFER_OVERFLOW);
if (dwRet == ERROR_BUFFER_OVERFLOW)
{

pAdapterAddresses = (IP_ADAPTER_ADDRESSES *) LocalAlloc(LMEM_ZEROINIT, dwSize);
if (! pAdapterAddresses)
{

rc = ARES_ENOMEM;
goto cleanup;
}
}

else
{
rc = ARES_ENODATA;
goto cleanup;

}

dwRet = (*GetAdaptersAddressesProc)(AF_UNSPEC, 0, NULL, pAdapterAddresses, &dwSize);

if (dwRet != ERROR_SUCCESS)
{
rc = ARES_ENODATA;
goto cleanup;

}

{
IP_ADAPTER_ADDRESSES * AI = NULL;

for (AI = pAdapterAddresses; AI != NULL; AI = AI->Next)
{
PIP_ADAPTER_DNS_SERVER_ADDRESS dnsServers = AI->FirstDnsServerAddress;

// find 'addr' in adapter's list of dns servers.
for (; dnsServers; dnsServers = dnsServers->Next)

{
if (! dnsServers->Address.lpSockaddr)
continue;

if (dnsServers->Address.lpSockaddr->sa_family == AF_INET)
{

struct sockaddr_in sockAddr = *(struct sockaddr_in*)(dnsServers->Address.lpSockaddr);
if (memcmp(&addr, &sockAddr.sin_addr.s_addr, sizeof(struct in_addr)) == 0)

{
*physAddrLen = AI->PhysicalAddressLength;
if (*physAddrLen > physicalAddrBufSz)

*physAddrLen = physicalAddrBufSz;
memcpy(physicalAddr, &AI->PhysicalAddress[0], *physAddrLen);

rc = ARES_SUCCESS;
goto cleanup;
}
}

}
}
}
rc = ARES_ENOTFOUND;

cleanup:
if (hLib)
FreeLibrary(hLib);

if (pAdapterAddresses)
LocalFree(pAdapterAddresses);
return rc;
#else // WIN32
return ARES_ENOTIMP;
#endif // WIN32
}

#ifdef WIN32
static int init_by_defaults_windows_nameservers_getadaptersaddresses(ares_channel channel)

{
/*
* Way of getting nameservers that should work on all Windows from 98 on.

*/
FIXED_INFO * FixedInfo;
ULONG ulOutBufLen;

DWORD dwRetVal;
IP_ADDR_STRING * pIPAddr;
HANDLE hLib;

int num;
DWORD (WINAPI *GetNetworkParams)(FIXED_INFO*, DWORD*);

hLib = LoadLibrary(TEXT("iphlpapi.dll"));
if(!hLib)

{
return ARES_ENOTIMP;
}

(void*)GetNetworkParams = GetProcAddress(hLib, TEXT("GetNetworkParams"));

if(!GetNetworkParams)
{
FreeLibrary(hLib);
return ARES_ENOTIMP;

}
//printf("ARES: figuring out DNS servers\n");
FixedInfo = (FIXED_INFO *) GlobalAlloc( GPTR, sizeof( FIXED_INFO ) );

ulOutBufLen = sizeof( FIXED_INFO );

if( ERROR_BUFFER_OVERFLOW == (*GetNetworkParams)( FixedInfo, &ulOutBufLen ) )

{
GlobalFree( FixedInfo );
FixedInfo = (FIXED_INFO *)GlobalAlloc( GPTR, ulOutBufLen );

}

if ( dwRetVal = (*GetNetworkParams)( FixedInfo, &ulOutBufLen ) )

{
//printf("ARES: couldn't get network params\n");
GlobalFree( FixedInfo );

FreeLibrary(hLib);
return ARES_ENODATA;
}

else
{
/**
printf( "Host Name: %s\n", FixedInfo -> HostName );

printf( "Domain Name: %s\n", FixedInfo -> DomainName );
printf( "DNS Servers:\n" );
printf( "\t%s\n", FixedInfo -> DnsServerList.IpAddress.String );
**/

// Count how many nameserver entries we have and allocate memory for them.
num = 0;
pIPAddr = &FixedInfo->DnsServerList;

while ( pIPAddr && strlen(pIPAddr->IpAddress.String) > 0)
{
num++;

pIPAddr = pIPAddr ->Next;
}
if(num>0)
{

channel->servers = malloc( (num) * sizeof(struct server_state));
if (!channel->servers)

{
GlobalFree( FixedInfo );

FreeLibrary(hLib);
return ARES_ENOMEM;

}
memset(channel->servers, '\0', num * sizeof(struct server_state));

channel->nservers = 0;
pIPAddr = &FixedInfo->DnsServerList;

while ( pIPAddr && strlen(pIPAddr->IpAddress.String) > 0)
{

struct in_addr addr;
addr.s_addr = inet_addr(pIPAddr->IpAddress.String);
// append unique only

if (find_server(channel->servers, channel->nservers, addr) == -1)
{
// printf( "ARES: %s\n", pIPAddr ->IpAddress.String );

#ifdef USE_IPV6
channel->servers[ channel->nservers ].family = AF_INET;

#endif
channel->servers[channel->nservers].addr = addr;
if ((channel->flags & ARES_FLAG_TRY_NEXT_SERVER_ON_RCODE3))

{
get_physical_address(channel->servers[channel->nservers].physical_addr,
MAX_ADAPTER_ADDRESS_LENGTH,

&channel->servers[channel->nservers].physical_addr_len,
addr);

}
channel->nservers++;
}

pIPAddr = pIPAddr ->Next;
}
//printf("ARES: got all %d nameservers\n",num);

}
else
{
/* If no specified servers, try a local named. */

channel->servers = malloc(sizeof(struct server_state));
if (!channel->servers)

return ARES_ENOMEM;
memset(channel->servers, '\0', sizeof(struct server_state));

#ifdef USE_IPV6
channel->servers[0].family = AF_INET;

#endif

channel->servers[0].addr.s_addr = htonl(INADDR_LOOPBACK);

channel->servers[0].default_localhost_server = 1;
channel->nservers = 1;

}

GlobalFree( FixedInfo );
FreeLibrary(hLib);

}
return ARES_SUCCESS;
}

On Wed, Feb 26, 2014 at 3:11 PM, Herbert Karajan <boost.regex@xxxxxxxxx> wrote:

I have more stuff here:

in ares_init.c file, if replace the function

"#ifdef WIN32

static int init_by_defaults_windows_nameservers_getadaptersaddresses(ares_channel channel)" in 1.9.x by same function which is in 1.8.x, then get it works, the issue disappear.

Best regards,

On Tue, Feb 25, 2014 at 11:15 PM, Herbert Karajan <boost.regex@xxxxxxxxx> wrote:

Note: I'm using the Win 8 and VC++9

On Tue, Feb 25, 2014 at 11:12 PM, Herbert Karajan <boost.regex@xxxxxxxxx> wrote:

Hi Scott, attached the 1.8.5 log,

I'm use the stackThread class as below:

mStackThread(mStack);
mStackThread.run();

I also created a DUM thread in my application:

void DumThread::thread()
{
while(!ThreadIf::isShutdown())

{
Lock lock(mDumMutex);
while (true)
{

if (!mDum.process())
{
break;
}

}
}
}

On Tue, Feb 25, 2014 at 10:52 PM, Scott Godin <sgodin@xxxxxxxxxxxxxxx> wrote:

To help speed up the investigation, can you post the log file when trying with 1.8.x? What OS/platform are you using? Can you also post your stack/dum processing loop (are you using EventStackThread or the old buildFdSet/process mechanism?

Thanks,
Scott

On Tue, Feb 25, 2014 at 4:50 AM, Herbert Karajan <boost.regex@xxxxxxxxx> wrote:

Hi, I got a "503 DNS timeout" error after I update my application to resip 1.9.0 and 1.9.1, it works fine after converted back to 1.8.x.

Please find attached the log file.

_______________________________________________
resiprocate-users mailing list
resiprocate-users@xxxxxxxxxxxxxxx
List Archive: http://list.resiprocate.org/archive/resiprocate-users/

References:
- [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Herbert Karajan
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Scott Godin
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Herbert Karajan
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Herbert Karajan
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Herbert Karajan
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Herbert Karajan
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Scott Godin
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Herbert Karajan
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Scott Godin
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Herbert Karajan
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Scott Godin
- Re: [reSIProcate-users] DNS timeout bug in 1.9.x
  - From: Scott Godin

Prev by Date: Re: [reSIProcate-users] DNS timeout bug in 1.9.x
Next by Date: [reSIProcate-users] Tracking resources to prevent memory exhaustion
Previous by thread: Re: [reSIProcate-users] DNS timeout bug in 1.9.x
Next by thread: [reSIProcate-users] Detected the network connection
Index(es):
- Date
- Thread