UPDATE: Using Multiple interfaces with KVM and Xen

I had a system crash with blinking keyboard lights. The error has something to do with either ACPI power states or the bridge interface. On my system setting br1 to DHCP causes some kind of issue when using DHCP, here is my log:

Feb 26 03:25:29 mattcom1 dhclient: DHCPREQUEST on br1 to port 67
Feb 26 03:25:29 mattcom1 ntpd[2188]: Listening on interface #22 eth0_rename, Enabled
Feb 26 03:25:29 mattcom1 avahi-daemon[2374]: Registering new address record for fe80::20a:5eff:fe45:7eca on eth1.*.
Feb 26 03:25:30 mattcom1 avahi-daemon[2374]: Registering new address record for fe80::20a:5eff:fe45:7eca on br1.*.
Feb 26 03:25:31 mattcom1 ntpd[2188]: Listening on interface #23 br1, fe80::20a:5eff:fe45:7eca#123 Enabled
Feb 26 03:25:32 mattcom1 dhclient: DHCPREQUEST on br1 to port 67
Feb 26 03:25:32 mattcom1 kernel: ————[ cut here ]————
Feb 26 03:25:32 mattcom1 kernel: WARNING: at net/core/dev.c:1505 skb_gso_segment+0×6e/0×153() (Tainted: P )
Feb 26 03:25:32 mattcom1 kernel: Hardware name:
Feb 26 03:25:32 mattcom1 kernel: Modules linked in: sit tunnel4 udf fuse bridge stp bnep sco l2cap bluetooth sunrpc ipv6 cpufreq_ondemand acpi
_cpufreq dm_multipath kvm_intel kvm uinput snd_ens1371 gameport snd_rawmidi snd_ac97_codec ac97_bus nvidia(P) snd_seq_dummy snd_seq_oss snd_se
q_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd iTCO_wdt e1000e iTCO_vendor_support soundcore ppdev 3c59x
firewire_ohci snd_page_alloc firewire_core i2c_i801 parport_pc mii floppy parport i2c_core pcspkr crc_itu_t ata_generic pata_acpi sha256_gener
ic cbc aes_i586 aes_generic dm_crypt crypto_blkcipher [last unloaded: microcode]

then later…

Feb 26 03:52:06 mattcom1 kernel: qemu-kvm[10261]: segfault at 1df ip 080afcb8 sp bfed6898 error 4 in qemu-kvm[8047000+175000]
Feb 26 03:52:06 mattcom1 avahi-daemon[2374]: Withdrawing address record for fe80::3c3b:13ff:fedb:cfcb on vnet0.
Feb 26 03:52:06 mattcom1 kernel: br1: port 2(vnet0) entering disabled state
Feb 26 03:52:06 mattcom1 kernel: device vnet0 left promiscuous mode
Feb 26 03:52:06 mattcom1 kernel: br1: port 2(vnet0) entering disabled state
Feb 26 03:52:07 mattcom1 ntpd[2188]: Deleting interface #27 vnet0, fe80::3c3b:13ff:fedb:cfcb#123, interface stats: received=0, sent=0, dropped
=0, active_time=1289 secs

So I set br1 to a static IP and seems to work, though since it is a bridge I don’t see why it needs an IP address anyways… so turn it off by editing the bridge and the real interface to have no IP.





Using Multiple interfaces with KVM and Xen

I’ve been meaning to write a post on ethernet bridges and how they can easily be used to accommodate virtual machines with their “own” physical NIC(Network Interface Card).  I see a ton of post online about people struggling to get multiple ethernet cards to work in Xen and other hypervisors like KVM.  A common complaint is that when both NIC cards are plugged into the LAN they lose connectivity from all machines including the host.

Many sites make an attempt to explain the problem of multiple interfaces on the same network by walking you through a Xen custom configuration.  However they fail to identify the concept of bridges, layer 2 loops and why Spanning Tree Protocol is your friend!  So many virtualization nuts(like myself) spend hours trying to find a problem with Xen, Vmware, KVM, whatever… when the problem may just be how the interfaces are configured.

The goals of this post:

* define ethernet bridging

* explain ethernet loops

* discuss how this relates to VM’s and the hypervisor

* LAB: set up two ethernet cards for guest VM and my Fedora 10 KVM Server



Configuring Asterisk for a remote Syslog Server PART I

Syslog & Rsyslog are the mainstay tools of event logging.  Every standard UNIX/Linux based Operating System comes with some version.  Asterisk logging capabilities allow for some or all events to be sent to syslogd for post processing; (that is) to store in a SQL database or for sending logs to a remote syslog server.

If you work on or troubleshoot Asterisk PBX servers than you know there are a many pieces to put together.  Putting together 30 Polycom IP SoundPoint phones, the network, and configuring Asterisk can a huge task.  I tell most Administrators to expect a “shake out” period.  Usually about for a week.

Start with the simplist feature set possible, and work your way out.  However, occasionally dial plan SNAFU’s aren’t noticed right away.  Like for instance when a user cannot dial a certain area code or there is typo in some utility extension.  By this time there might be wide use of your(mostly operational) Asterisk PBX, making it diffacult to watch the CLI and catch the error.  For the record I’m using Rsyslog, Asterisk 1.4.23-rc2 on Fedora 10 i386, 32bit, Intel yada yada… We’ll start by editing the logger.conf file.


; Logging Configuration
; In this file, you configure logging to files or to
; the syslog system.
; “logger reload” at the CLI will reload configuration
; of the logging system.

; Customize the display of debug message time stamps
; this example is the ISO 8601 date format (yyyy-mm-dd HH:MM:SS)
; see strftime(3) Linux manual for format specifiers
;dateformat=%F %T
; This appends the hostname to the name of the log files.
;appendhostname = yes
; This determines whether or not we log queue events to a file
; (defaults to yes).
;queue_log = no
; This determines whether or not we log generic events to a file
; (defaults to yes).
;event_log = no
; For each file, specify what to log.
; For console logging, you set options at start of
; Asterisk with -v for verbose and -d for debug
; See ‘asterisk -h’ for more information.
; Directory for log files is configures in asterisk.conf
; option astlogdir
; Format is “filename” and then “levels” of debugging to be included:
;    debug
;    notice
;    warning
;    error
;    verbose
;    dtmf
; Special filename “console” represents the system console
; We highly recommend that you DO NOT turn on debug mode if you are simply
; running a production system.  Debug mode turns on a LOT of extra messages,
; most of which you are unlikely to understand without an understanding of
; the underlying code.  Do NOT report debug messages as code issues, unless
; you have a specific issue that you are attempting to debug.  They are
; messages for just that — debugging — and do not rise to the level of
; something that merit your attention as an Asterisk administrator.  Debug
; messages are also very verbose and can and do fill up logfiles quickly;
; this is another reason not to have debug mode on a production system unless
; you are in the process of debugging a specific issue.
;debug => debug
console => notice,warning,error
;console => notice,warning,error,debug
messages => notice,warning,error
;full => notice,warning,error,debug,verbose

;syslog keyword : This special keyword logs to syslog facility

syslog.local1 => verbose

The first time I looked at this I was confused, ‘ ;syslog keyword : This special keyword logs to syslog facility ‘.  This almost implies that ‘keyword’ could be anything.  So what if i call it

/etc/asterisk/logger.conf :

;syslog.local1 => verbose

; replace with

syslog.asterisk => verbose

Now I’ll create a matching entry in the /etc/rsyslog.conf (Fedora 10 uses rsyslog).  Keep in mind you may use /etc/syslog.conf on your system.


#rsyslog v3 config file

# if you experience problems, check
# http://www.rsyslog.com/troubleshoot for assistance

#### MODULES ####

$ModLoad imuxsock.so    # provides support for local system logging (e.g. via logger command)
$ModLoad imklog.so    # provides kernel logging support (previously done by rklogd)
#$ModLoad immark.so    # provides –MARK– message capability

# Provides UDP syslog reception
$ModLoad imudp.so
$UDPServerRun 514

# Provides TCP syslog reception
$ModLoad imtcp.so
$InputTCPServerRun 514


# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don’t log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog

# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 *

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log


# Below you will see series of expressions to catch asterisk logs

#Some work, other expressions will not.  Why is that?


local0.*                        /var/log/asterisk.log

local1.*                        /var/log/local-cli.log

#The asterisk ‘logger.conf’ file matches entries ‘local1’ and ‘asterisk’.

asterisk.*                    /var/log/asterisk-cli.log


# ### begin forwarding rule ###
# The statement between the begin … end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$WorkDirectory /var/spppl/rsyslog # where to place spool files
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g., port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###