<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	xmlns:creativeCommons="http://backend.userland.com/creativeCommonsRssModule">

<channel>
	<title>Fabian Moser &#187; Python</title>
	<atom:link href="http://fabianmoser.at/schlagwort/python/feed/" rel="self" type="application/rss+xml" />
	<link>http://fabianmoser.at</link>
	<description>&#34;as simple as possible, but not simpler&#34;</description>
	<lastBuildDate>Thu, 26 Jan 2012 13:19:36 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
<creativeCommons:license>http://creativecommons.org/licenses/by-sa/3.0/at/</creativeCommons:license>		<item>
		<title>More parrallel programming</title>
		<link>http://fabianmoser.at/blog/2011/08/19/more-parrallel-programming/</link>
		<comments>http://fabianmoser.at/blog/2011/08/19/more-parrallel-programming/#comments</comments>
		<pubDate>Fri, 19 Aug 2011 20:01:57 +0000</pubDate>
		<dc:creator>Fabian Moser</dc:creator>
				<category><![CDATA[Programmieren]]></category>
		<category><![CDATA[Concurrency]]></category>
		<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://fabianmoser.at/?p=1114</guid>
		<description><![CDATA[As a faithful follower of the Fedora Planet, today I stumbled upon a post about parallel programming in Python. Having made similar experiences myself, I would like to add another alternative for parallel programming in Python. I could have posted &#8230; <a href="http://fabianmoser.at/blog/2011/08/19/more-parrallel-programming/">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<p>As a faithful follower of the <a href="http://planet.fedoraproject.org/">Fedora Planet</a>, today I stumbled upon a <a href="http://blog.pingoured.fr/index.php?post/2011/08/19/Parrallel-programming-in-python">post about parallel programming in Python</a>. Having made similar experiences myself, I would like to add another alternative for parallel programming in Python. I could have posted this in the comments of the original post, but this way the formatting is nicer.</p>
<p>My point is, that <a href="http://www.parallelpython.com/">Parallel Python</a> is a really nice library, but the functionality (at least at the level demonstrated here) is also provided by <a href="http://docs.python.org/library/multiprocessing.html">the <code>multiprocessing</code> module included with Python</a>.</p>
<p>Here is my slightly modified implementation of the same program:</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #808080; font-style: italic;">#!/usr/bin/python</span>
&nbsp;
<span style="color: #483d8b;">&quot;&quot;&quot;
Another asynchronous python example
&quot;&quot;&quot;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">import</span> multiprocessing
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">time</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> background_stuff<span style="color: black;">&#40;</span>num<span style="color: black;">&#41;</span>:
  <span style="color: #dc143c;">time</span>.<span style="color: black;">sleep</span><span style="color: black;">&#40;</span><span style="color: #ff4500;">5</span><span style="color: black;">&#41;</span>
  <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #483d8b;">&quot;%s I'm done&quot;</span> <span style="color: #66cc66;">%</span> num
&nbsp;
<span style="color: #ff7700;font-weight:bold;">if</span> __name__ == <span style="color: #483d8b;">&quot;__main__&quot;</span>:
    <span style="color: #ff7700;font-weight:bold;">print</span> <span style="color: #483d8b;">&quot;Start at:&quot;</span> , <span style="color: #dc143c;">time</span>.<span style="color: black;">asctime</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">time</span>.<span style="color: black;">localtime</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
    pool = multiprocessing.<span style="color: black;">Pool</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">print</span> <span style="color: #483d8b;">&quot;Start doing something&quot;</span>
    it = pool.<span style="color: black;">imap</span><span style="color: black;">&#40;</span>background_stuff, <span style="color: black;">&#91;</span><span style="color: black;">&#40;</span><span style="color: #ff4500;">1</span>,<span style="color: black;">&#41;</span>, <span style="color: black;">&#40;</span><span style="color: #ff4500;">2</span>,<span style="color: black;">&#41;</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">print</span> <span style="color: #483d8b;">&quot;Do something...&quot;</span>
    <span style="color: #ff7700;font-weight:bold;">print</span> <span style="color: #483d8b;">&quot; ... do something else...&quot;</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">print</span> it.<span style="color: black;">next</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">print</span> it.<span style="color: black;">next</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">print</span> <span style="color: #483d8b;">&quot;End at:&quot;</span>, <span style="color: #dc143c;">time</span>.<span style="color: black;">asctime</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">time</span>.<span style="color: black;">localtime</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span><span style="color: black;">&#41;</span></pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://fabianmoser.at/blog/2011/08/19/more-parrallel-programming/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>How-to reduce fail2ban memory usage</title>
		<link>http://fabianmoser.at/blog/2009/05/15/how-to-reduce-fail2ban-memory-usage/</link>
		<comments>http://fabianmoser.at/blog/2009/05/15/how-to-reduce-fail2ban-memory-usage/#comments</comments>
		<pubDate>Fri, 15 May 2009 11:14:05 +0000</pubDate>
		<dc:creator>Fabian Moser</dc:creator>
				<category><![CDATA[Server]]></category>
		<category><![CDATA[HOWTO]]></category>
		<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://www.fabianmoser.at/?p=387</guid>
		<description><![CDATA[This morning, when I did the routinely scan of the server&#8217;s resource usage history, I noticed a suspicious network activity between 1 and 5 am. Some reading of the latest log files soon identified the traffic to have been caused &#8230; <a href="http://fabianmoser.at/blog/2009/05/15/how-to-reduce-fail2ban-memory-usage/">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<p>This morning, when I did the routinely scan of the server&#8217;s resource usage history, I noticed a suspicious network activity between 1 and 5 am. Some reading of the latest log files soon identified the traffic to have been caused by a dictionary attack on my SSH server. I took the opportunity to extend my current setup for the <a href="http://debaday.debian.net/2007/04/29/fail2ban-an-enemy-of-script-kiddies/">script-kiddie enemy</a> called <a href="http://www.fail2ban.org">fail2ban</a>. This program monitors potentially any service&#8217;s log file for failed login attempts and if their number exceeds a certain limit, it blocks the issuing host using <a href="http://www.netfilter.org/">iptables</a> rules.</p>
<p>Unfortunately the first start of the new service turned out to blow up the memory usage by about 100 MB which is unacceptable regarding the tight resources of my virtual private server. As I found out, <a href="http://www.supersoftcafe.com/2009/04/09/vps-configuration">others</a> had similar experience and switched to <a href="http://denyhosts.sourceforge.net/">DenyHosts</a> due to this issue. My experience with setting up <a href="http://trac.edgewall.org/">Trac</a> two weeks ago taught me that a Python application (like fain2ban) might consume a lot of memory only because of the relatively oversized default stack size on Linux.</p>
<p>The means to reduce the default stack size in Linux are widely known to be the limits.conf file and the ulimit command. But how to use those two in my situation? The solution turns out to be a one-liner on Debian Lenny: All I had to do was to append the ulimit command to my /etc/default/fail2ban file.</p>
<p>This is the changed /etc/default/fail2ban file:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;"># This file is part of Fail2Ban.</span>
<span style="color: #666666; font-style: italic;">#</span>
<span style="color: #666666; font-style: italic;"># Fail2Ban is free software; you can redistribute it and/or modify</span>
<span style="color: #666666; font-style: italic;"># it under the terms of the GNU General Public License as published by</span>
<span style="color: #666666; font-style: italic;"># the Free Software Foundation; either version 2 of the License, or</span>
<span style="color: #666666; font-style: italic;"># (at your option) any later version.</span>
<span style="color: #666666; font-style: italic;">#</span>
<span style="color: #666666; font-style: italic;"># Fail2Ban is distributed in the hope that it will be useful,</span>
<span style="color: #666666; font-style: italic;"># but WITHOUT ANY WARRANTY; without even the implied warranty of</span>
<span style="color: #666666; font-style: italic;"># MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the</span>
<span style="color: #666666; font-style: italic;"># GNU General Public License for more details.</span>
<span style="color: #666666; font-style: italic;">#</span>
<span style="color: #666666; font-style: italic;"># You should have received a copy of the GNU General Public License</span>
<span style="color: #666666; font-style: italic;"># along with Fail2Ban; if not, write to the Free Software</span>
<span style="color: #666666; font-style: italic;"># Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA</span>
<span style="color: #666666; font-style: italic;">#</span>
<span style="color: #666666; font-style: italic;"># Author: Cyril Jaquier</span>
<span style="color: #666666; font-style: italic;">#</span>
<span style="color: #666666; font-style: italic;"># $Revision: 1.2 $</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Command line options for Fail2Ban. Refer to &quot;fail2ban-client -h&quot; for</span>
<span style="color: #666666; font-style: italic;"># valid options.</span>
&nbsp;
<span style="color: #007800;">FAIL2BAN_OPTS</span>=<span style="color: #ff0000;">&quot;&quot;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">ulimit</span> <span style="color: #660033;">-s</span> <span style="color: #000000;">256</span></pre></div></div>

<p>Using this sets the default stack size for the Python instances running fail2ban to 256 KB and lowers the memory consumption of fail2ban approximately by a factor of 10.</p>
]]></content:encoded>
			<wfw:commentRss>http://fabianmoser.at/blog/2009/05/15/how-to-reduce-fail2ban-memory-usage/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Find overlapping polygons with Python</title>
		<link>http://fabianmoser.at/blog/2009/04/30/find-overlapping-polygons-with-python/</link>
		<comments>http://fabianmoser.at/blog/2009/04/30/find-overlapping-polygons-with-python/#comments</comments>
		<pubDate>Thu, 30 Apr 2009 09:11:10 +0000</pubDate>
		<dc:creator>Fabian Moser</dc:creator>
				<category><![CDATA[Arbeit]]></category>
		<category><![CDATA[Software]]></category>
		<category><![CDATA[Geometry]]></category>
		<category><![CDATA[Pylab]]></category>
		<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://www.fabianmoser.at/?p=293</guid>
		<description><![CDATA[Yesterday I was confronted with a seemingly simple problem: how can one find out if the rectangles or, more general, polygons on a surface are overlapping or not? Surprisingly, because they have an amazing collection of tools, the matplotlib library &#8230; <a href="http://fabianmoser.at/blog/2009/04/30/find-overlapping-polygons-with-python/">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<p>Yesterday I was confronted with a seemingly simple problem: how can one find out if the rectangles or, more general, polygons on a surface are overlapping or not? Surprisingly, because they have an amazing collection of tools, the <a href="http://matplotlib.sourceforge.net/">matplotlib</a> library used in this context to actually draw the rectangles, doesn&#8217;t seem to have this kind of functionality. Of course there are loads of examples on the web for 2D collision detection, but I couldn&#8217;t find one written in Python. But nevertheless, I found <a href="http://www.cs.wustl.edu/~pless/546/lectures/lecture21.pdf">lecture notes</a> on <a href="http://www.cs.wustl.edu/~pless/">Robert Pless&#8217;s website</a>, which taught me the quadrant method to check if one point lies inside a polygon. Then it is only a small step to find if polygons are intersecting.</p>
<p>After writing a Python module providing the functionality, it naturally turned out to be rather slow on big numbers of polygons to be checked. So I continued to write a function which checks which polygons in a given set are overlapping or touching. To use the resources on my machine efficiently, I made two versions of this later function: one for conservative, serial processing and a second which uses <a href="http://www.parallelpython.com/">Parallel Python</a> to distribute the workload among the CPUs found in the system.</p>
<p>If someone else needs this kind of functionaltiy as well or simply is interested in how I did it, here is the interesting part. The whole file with unit tests and documentation can be downloaded as well: <a href="http://www.fabianmoser.at/files/2009/04/polygons_overlapping.py">polygons_overlapping.py</a></p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">import</span> pylab
&nbsp;
<span style="color: #ff7700;font-weight:bold;">class</span> PolygonsTouching<span style="color: black;">&#40;</span> <span style="color: #008000;">Exception</span> <span style="color: black;">&#41;</span>:
    <span style="color: #483d8b;">&quot;&quot;&quot; This exception is triggered when two polygons touch at one point.
&nbsp;
    This is for internal use only and will be caught before returning.
&nbsp;
    &quot;&quot;&quot;</span>
    <span style="color: #ff7700;font-weight:bold;">def</span> <span style="color: #0000cd;">__init__</span><span style="color: black;">&#40;</span> <span style="color: #008000;">self</span>, x=<span style="color: #ff4500;">0</span>, y=<span style="color: #ff4500;">0</span> <span style="color: black;">&#41;</span>:
        <span style="color: #008000;">self</span>.<span style="color: black;">x</span>, <span style="color: #008000;">self</span>.<span style="color: black;">y</span> = x, y
    <span style="color: #ff7700;font-weight:bold;">def</span> <span style="color: #0000cd;">__str__</span><span style="color: black;">&#40;</span> <span style="color: #008000;">self</span> <span style="color: black;">&#41;</span>:
        <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #483d8b;">'The tested polygons at least touch each other at (%f,%f)'</span>\
               <span style="color: #66cc66;">%</span> <span style="color: black;">&#40;</span> <span style="color: #008000;">self</span>.<span style="color: black;">x</span>, <span style="color: #008000;">self</span>.<span style="color: black;">y</span> <span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">def</span> shift<span style="color: black;">&#40;</span> <span style="color: #008000;">self</span>, dx, dy <span style="color: black;">&#41;</span>:
        <span style="color: #008000;">self</span>.<span style="color: black;">x</span> += dx
        <span style="color: #008000;">self</span>.<span style="color: black;">y</span> += dy
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> pair_overlapping<span style="color: black;">&#40;</span> polygon1, polygon2, digits = <span style="color: #008000;">None</span> <span style="color: black;">&#41;</span>:
    <span style="color: #483d8b;">&quot;&quot;&quot; Find out if polygons are overlapping or touching.
&nbsp;
    The function makes use of the quadrant method to find out if a point is
    inside a given polygon.
&nbsp;
    polygon1, polygon2 -- Two arrays of [x,y] pairs where the last and the
        first pair is the same, because the polygon has to be closed.
    digits -- The number of digits relevant for the decision between
        separate and touching or touching and overlapping
&nbsp;
    Returns 0 if the given polygons are neither overlapping nor touching,
    returns 1 if they are not overlapping, but touching and
    returns 2 if they are overlapping
&nbsp;
    &quot;&quot;&quot;</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> calc_walk_summand<span style="color: black;">&#40;</span> r1, r2, digits = <span style="color: #008000;">None</span> <span style="color: black;">&#41;</span>:
        <span style="color: #483d8b;">&quot;&quot;&quot; Calculates the summand along one edge depending on axis crossings.
&nbsp;
        Follows the edge between two points and checks if one or both axes are
        being crossed. If They are crossed in clockwise sense, it returns +1
        otherwise -1. Going through the origin raises the PolygonsTouching
        exception.
&nbsp;
        Returns one of -2, -1, 0, +1, +2 or raises PolygonsTouching
&nbsp;
        &quot;&quot;&quot;</span>
        x, y = <span style="color: #ff4500;">0</span>, <span style="color: #ff4500;">1</span> <span style="color: #808080; font-style: italic;"># indices for better readability</span>
        summand = <span style="color: #ff4500;">0</span> <span style="color: #808080; font-style: italic;"># the return value</span>
        tx, ty = <span style="color: #008000;">None</span>, <span style="color: #008000;">None</span> <span style="color: #808080; font-style: italic;"># on division by zero, set parameters to None</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> r1<span style="color: black;">&#91;</span>x<span style="color: black;">&#93;</span> <span style="color: #66cc66;">!</span>= r2<span style="color: black;">&#91;</span>x<span style="color: black;">&#93;</span>:
            ty = r1<span style="color: black;">&#91;</span>x<span style="color: black;">&#93;</span> / <span style="color: black;">&#40;</span> r1<span style="color: black;">&#91;</span>x<span style="color: black;">&#93;</span> - r2<span style="color: black;">&#91;</span>x<span style="color: black;">&#93;</span> <span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># where it crosses the y axis</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> r1<span style="color: black;">&#91;</span>y<span style="color: black;">&#93;</span> <span style="color: #66cc66;">!</span>= r2<span style="color: black;">&#91;</span>y<span style="color: black;">&#93;</span>:
            tx = r1<span style="color: black;">&#91;</span>y<span style="color: black;">&#93;</span> / <span style="color: black;">&#40;</span> r1<span style="color: black;">&#91;</span>y<span style="color: black;">&#93;</span> - r2<span style="color: black;">&#91;</span>y<span style="color: black;">&#93;</span> <span style="color: black;">&#41;</span> <span style="color: #808080; font-style: italic;"># where it crosses the x axis</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> tx == <span style="color: #008000;">None</span>: tx = ty
        <span style="color: #ff7700;font-weight:bold;">if</span> ty == <span style="color: #008000;">None</span>: ty = tx
        rsign = pylab.<span style="color: black;">sign</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> digits <span style="color: #66cc66;">!</span>= <span style="color: #008000;">None</span>:
            rsign = <span style="color: #ff7700;font-weight:bold;">lambda</span> x: pylab.<span style="color: black;">sign</span><span style="color: black;">&#40;</span> <span style="color: #008000;">round</span><span style="color: black;">&#40;</span> x, digits <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span>
        sign_x = rsign<span style="color: black;">&#40;</span> r1<span style="color: black;">&#91;</span>x<span style="color: black;">&#93;</span> + tx <span style="color: #66cc66;">*</span> <span style="color: black;">&#40;</span> r2<span style="color: black;">&#91;</span>x<span style="color: black;">&#93;</span> - r1<span style="color: black;">&#91;</span>x<span style="color: black;">&#93;</span> <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span>
        sign_y = rsign<span style="color: black;">&#40;</span> r1<span style="color: black;">&#91;</span>y<span style="color: black;">&#93;</span> + ty <span style="color: #66cc66;">*</span> <span style="color: black;">&#40;</span> r2<span style="color: black;">&#91;</span>y<span style="color: black;">&#93;</span> - r1<span style="color: black;">&#91;</span>y<span style="color: black;">&#93;</span> <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: black;">&#40;</span> tx <span style="color: #66cc66;">&gt;</span>= <span style="color: #ff4500;">0</span> <span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">and</span> <span style="color: black;">&#40;</span> tx <span style="color: #66cc66;">&lt;</span> <span style="color: #ff4500;">1</span> <span style="color: black;">&#41;</span>:
            <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: black;">&#40;</span> sign_x == <span style="color: #ff4500;">0</span> <span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">and</span> <span style="color: black;">&#40;</span> sign_y == <span style="color: #ff4500;">0</span> <span style="color: black;">&#41;</span>:
                <span style="color: #ff7700;font-weight:bold;">raise</span> PolygonsTouching<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
            summand += sign_x <span style="color: #66cc66;">*</span> pylab.<span style="color: black;">sign</span><span style="color: black;">&#40;</span> r2<span style="color: black;">&#91;</span>y<span style="color: black;">&#93;</span> - r1<span style="color: black;">&#91;</span>y<span style="color: black;">&#93;</span> <span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: black;">&#40;</span> ty <span style="color: #66cc66;">&gt;</span>= <span style="color: #ff4500;">0</span> <span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">and</span> <span style="color: black;">&#40;</span> ty <span style="color: #66cc66;">&lt;</span> <span style="color: #ff4500;">1</span> <span style="color: black;">&#41;</span>:
            <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: black;">&#40;</span> sign_x == <span style="color: #ff4500;">0</span> <span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">and</span> <span style="color: black;">&#40;</span> sign_y == <span style="color: #ff4500;">0</span> <span style="color: black;">&#41;</span>:
                <span style="color: #ff7700;font-weight:bold;">raise</span> PolygonsTouching<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
            summand += sign_y <span style="color: #66cc66;">*</span> pylab.<span style="color: black;">sign</span><span style="color: black;">&#40;</span> r1<span style="color: black;">&#91;</span>x<span style="color: black;">&#93;</span> - r2<span style="color: black;">&#91;</span>x<span style="color: black;">&#93;</span> <span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">return</span> summand
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> current_and_next<span style="color: black;">&#40;</span> iterable <span style="color: black;">&#41;</span>:
        <span style="color: #483d8b;">&quot;&quot;&quot; Returns an iterator for each element and its following element.
&nbsp;
        &quot;&quot;&quot;</span>
        iterator = <span style="color: #008000;">iter</span><span style="color: black;">&#40;</span> iterable <span style="color: black;">&#41;</span>
        item = iterator.<span style="color: black;">next</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">for</span> next <span style="color: #ff7700;font-weight:bold;">in</span> iterator:
            <span style="color: #ff7700;font-weight:bold;">yield</span> <span style="color: black;">&#40;</span> item, next <span style="color: black;">&#41;</span>
            item = next
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> point_in_polygon<span style="color: black;">&#40;</span> xy, xyarray, digits = <span style="color: #008000;">None</span> <span style="color: black;">&#41;</span>:
        <span style="color: #483d8b;">&quot;&quot;&quot; Checks if a point lies inside a polygon using the quadrant method.
&nbsp;
        This moves the given point to the origin and shifts the polygon
        accordingly. Then for each edge of the polygon, calc_walk_summand is
        called. If the sum of all returned values from these calls is +4 or -4,
        the point lies indeed inside the polygon. Otherwise, if a
        PolygonsTouching exception has been caught, the point lies on ond of
        the edges of the polygon.
&nbsp;
        Returns the number of nodes of the polygon, if the point lies inside,
        otherwise 1 if the point lies on the polygon and if not, 0.
&nbsp;
        &quot;&quot;&quot;</span>
        moved = xyarray - xy <span style="color: #808080; font-style: italic;"># move currently checked point to the origin (0,0)</span>
        touching = <span style="color: #008000;">False</span> <span style="color: #808080; font-style: italic;"># this is used only if no overlap is found</span>
        walk_sum = <span style="color: #ff4500;">0</span>
        <span style="color: #ff7700;font-weight:bold;">for</span> cnxy <span style="color: #ff7700;font-weight:bold;">in</span> current_and_next<span style="color: black;">&#40;</span> moved <span style="color: black;">&#41;</span>:
            <span style="color: #ff7700;font-weight:bold;">try</span>:
                walk_sum += calc_walk_summand<span style="color: black;">&#40;</span> cnxy<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span>, cnxy<span style="color: black;">&#91;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span>, digits <span style="color: black;">&#41;</span>
            <span style="color: #ff7700;font-weight:bold;">except</span> PolygonsTouching, <span style="color: black;">&#40;</span>e<span style="color: black;">&#41;</span>:
                e.<span style="color: black;">shift</span><span style="color: black;">&#40;</span> <span style="color: #66cc66;">*</span>xy <span style="color: black;">&#41;</span>
                touching = <span style="color: #008000;">True</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: black;">&#40;</span> <span style="color: #008000;">abs</span><span style="color: black;">&#40;</span> walk_sum <span style="color: black;">&#41;</span> == <span style="color: #ff4500;">4</span> <span style="color: black;">&#41;</span>:
            <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">len</span><span style="color: black;">&#40;</span> xyarray <span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">elif</span> touching:
            <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #ff4500;">1</span>
        <span style="color: #ff7700;font-weight:bold;">else</span>:
            <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #ff4500;">0</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> polygons_overlapping<span style="color: black;">&#40;</span> p1, p2, digits = <span style="color: #008000;">None</span> <span style="color: black;">&#41;</span>:
        <span style="color: #483d8b;">&quot;&quot;&quot; Checks if one of the nodes of p1 lies inside p2.
&nbsp;
        This repeatedly calls point_in_polygon for each point of polygon p1
        and immediately returns if it is the case, because then the polygons
        are obviously overlapping.
&nbsp;
        Returns 2 for overlapping polygons, 1 for touching polygons and 0
        otherwise.
&nbsp;
        &quot;&quot;&quot;</span>
        degree_of_contact = <span style="color: #ff4500;">0</span>
        xyarrays = <span style="color: black;">&#91;</span> p1, p2 <span style="color: black;">&#93;</span>
        <span style="color: #ff7700;font-weight:bold;">for</span> xy <span style="color: #ff7700;font-weight:bold;">in</span> xyarrays<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span>:
            degree_of_contact += point_in_polygon<span style="color: black;">&#40;</span> xy, xyarrays<span style="color: black;">&#91;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span>, digits <span style="color: black;">&#41;</span>
            <span style="color: #ff7700;font-weight:bold;">if</span> degree_of_contact <span style="color: #66cc66;">&gt;</span>= <span style="color: #008000;">len</span><span style="color: black;">&#40;</span> xyarrays<span style="color: black;">&#91;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#93;</span> <span style="color: black;">&#41;</span>:
                <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #ff4500;">2</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> degree_of_contact <span style="color: #66cc66;">&gt;</span> <span style="color: #ff4500;">0</span>:
            <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #ff4500;">1</span>
        <span style="color: #ff7700;font-weight:bold;">else</span>:
            <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #ff4500;">0</span>
&nbsp;
    way1 = polygons_overlapping<span style="color: black;">&#40;</span> polygon1, polygon2, digits <span style="color: black;">&#41;</span>
    way2 = <span style="color: #ff4500;">0</span>
    <span style="color: #ff7700;font-weight:bold;">if</span> way1 <span style="color: #66cc66;">&lt;</span> <span style="color: #ff4500;">2</span>: <span style="color: #808080; font-style: italic;"># Only if the polygons are not already found to be overlapping</span>
        way2 = polygons_overlapping<span style="color: black;">&#40;</span> polygon2, polygon1, digits <span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">max</span><span style="color: black;">&#40;</span> way1, way2 <span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> collection_overlapping_serial<span style="color: black;">&#40;</span> polygons, digits = <span style="color: #008000;">None</span> <span style="color: black;">&#41;</span>:
    <span style="color: #483d8b;">&quot;&quot;&quot; Similar to the collection_overlapping function, but forces serial
    processing.
&nbsp;
    &quot;&quot;&quot;</span>
    result = <span style="color: black;">&#91;</span><span style="color: black;">&#93;</span>
    pickle_polygons = <span style="color: black;">&#91;</span>p.<span style="color: black;">get_xy</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">for</span> p <span style="color: #ff7700;font-weight:bold;">in</span> polygons<span style="color: black;">&#93;</span>
    <span style="color: #ff7700;font-weight:bold;">for</span> i <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">xrange</span><span style="color: black;">&#40;</span> <span style="color: #008000;">len</span><span style="color: black;">&#40;</span> polygons <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span>:
        <span style="color: #ff7700;font-weight:bold;">for</span> j <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">xrange</span><span style="color: black;">&#40;</span> i+<span style="color: #ff4500;">1</span>, <span style="color: #008000;">len</span><span style="color: black;">&#40;</span> polygons <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span>:
            result.<span style="color: black;">append</span><span style="color: black;">&#40;</span> <span style="color: black;">&#40;</span> i, j, \
                pair_overlapping<span style="color: black;">&#40;</span> pickle_polygons<span style="color: black;">&#91;</span>i<span style="color: black;">&#93;</span>, pickle_polygons<span style="color: black;">&#91;</span>j<span style="color: black;">&#93;</span>, \
                                  digits <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">return</span> result
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> __cop_bigger_job<span style="color: black;">&#40;</span> polygons, index, digits = <span style="color: #008000;">None</span> <span style="color: black;">&#41;</span>:
    <span style="color: #483d8b;">&quot;&quot;&quot; This is a helper to efficiently distribute workload among processors.
&nbsp;
    &quot;&quot;&quot;</span>
    result = <span style="color: black;">&#91;</span><span style="color: black;">&#93;</span>
    <span style="color: #ff7700;font-weight:bold;">for</span> j <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">xrange</span><span style="color: black;">&#40;</span> index + <span style="color: #ff4500;">1</span>, <span style="color: #008000;">len</span><span style="color: black;">&#40;</span> polygons <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span>:
        result.<span style="color: black;">append</span><span style="color: black;">&#40;</span> <span style="color: black;">&#40;</span> index, j, \
            pair_overlapping<span style="color: black;">&#40;</span> polygons<span style="color: black;">&#91;</span>index<span style="color: black;">&#93;</span>, polygons<span style="color: black;">&#91;</span>j<span style="color: black;">&#93;</span>, digits <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">return</span> result
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> collection_overlapping_parallel<span style="color: black;">&#40;</span> polygons, digits = <span style="color: #008000;">None</span>, \
        ncpus = <span style="color: #483d8b;">'autodetect'</span> <span style="color: black;">&#41;</span>:
    <span style="color: #483d8b;">&quot;&quot;&quot; Like collection_overlapping, but forces parallel processing.
&nbsp;
    This function crashes if Parallel Python is not found on the system.
&nbsp;
    &quot;&quot;&quot;</span>
    <span style="color: #ff7700;font-weight:bold;">import</span> pp
    ppservers = <span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
    job_server = pp.<span style="color: black;">Server</span><span style="color: black;">&#40;</span> ncpus, ppservers=ppservers <span style="color: black;">&#41;</span>
    pickle_polygons = <span style="color: black;">&#91;</span>p.<span style="color: black;">get_xy</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span> <span style="color: #ff7700;font-weight:bold;">for</span> p <span style="color: #ff7700;font-weight:bold;">in</span> polygons<span style="color: black;">&#93;</span>
    jobs = <span style="color: black;">&#91;</span><span style="color: black;">&#93;</span>
    <span style="color: #ff7700;font-weight:bold;">for</span> i <span style="color: #ff7700;font-weight:bold;">in</span> <span style="color: #008000;">xrange</span><span style="color: black;">&#40;</span> <span style="color: #008000;">len</span><span style="color: black;">&#40;</span> polygons <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span>:
        job = job_server.<span style="color: black;">submit</span><span style="color: black;">&#40;</span> __cop_bigger_job, \
                                 <span style="color: black;">&#40;</span> pickle_polygons, i, digits, <span style="color: black;">&#41;</span>, \
                                 <span style="color: black;">&#40;</span> pair_overlapping, PolygonsTouching, <span style="color: black;">&#41;</span>, \
                                 <span style="color: black;">&#40;</span> <span style="color: #483d8b;">&quot;pylab&quot;</span>, <span style="color: black;">&#41;</span> <span style="color: black;">&#41;</span>
        jobs.<span style="color: black;">append</span><span style="color: black;">&#40;</span> job <span style="color: black;">&#41;</span>
    result = <span style="color: black;">&#91;</span><span style="color: black;">&#93;</span>
    <span style="color: #ff7700;font-weight:bold;">for</span> job <span style="color: #ff7700;font-weight:bold;">in</span> jobs:
        result += job<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
    <span style="color: #808080; font-style: italic;">#job_server.print_stats()</span>
    <span style="color: #ff7700;font-weight:bold;">return</span> result
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> collection_overlapping<span style="color: black;">&#40;</span> polygons, digits = <span style="color: #008000;">None</span> <span style="color: black;">&#41;</span>:
    <span style="color: #483d8b;">&quot;&quot;&quot; Look for pair-wise overlaps in a given list of polygons.
&nbsp;
    The function makes use of the quadrant method to find out if a point is
    inside a given polygon. It invokes the pair_overlapping function for each
    combination and produces and array of index pairs of these combinations
    together with the overlap number of that pair. The overlap number is 0 for
    no overlap, 1 for touching and 2 for overlapping polygons.
&nbsp;
    This function automatically selects between a serial and a parallel
    implementation of the search depending on whether Parallel Python is
    installed and can be imported or not.
&nbsp;
    polygons -- A list of arrays of [x,y] pairs where the last and the first
        pair of each array in the list is the same, because the polygons have
        to be closed.
    digits -- The number of digits relevant for the decision between
        separate and touching or touching and overlapping polygons.
&nbsp;
    Returns a list of 3-tuples
&nbsp;
    &quot;&quot;&quot;</span>
    <span style="color: #ff7700;font-weight:bold;">try</span>:
        <span style="color: #ff7700;font-weight:bold;">import</span> pp <span style="color: #808080; font-style: italic;"># try if parallel python is installed</span>
    <span style="color: #ff7700;font-weight:bold;">except</span> <span style="color: #008000;">ImportError</span>:
        <span style="color: #ff7700;font-weight:bold;">return</span> collection_overlapping_serial<span style="color: black;">&#40;</span> polygons, digits <span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">else</span>:
        <span style="color: #ff7700;font-weight:bold;">return</span> collection_overlapping_parallel<span style="color: black;">&#40;</span> polygons, digits <span style="color: black;">&#41;</span></pre></div></div>

</pre>
]]></content:encoded>
			<wfw:commentRss>http://fabianmoser.at/blog/2009/04/30/find-overlapping-polygons-with-python/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

