License Creative Commons Attribution Non-Commercial Share Alike 3.0-US
Keywords
operator overloading (1) Python (6) python tricks (2) tricks (7) underscore methods (2)
Permissions
Group Owner: iCapsid
Viewable by Everyone
Editable by All Siafoo Users
Note: You are viewing an old version of this article. View Latest Version

Python __Underscore__ Methods Atom Feed 1

Need to override an operator in your Python class? Ever wonder what all those double-underscore class methods do? Here's your answer. Note that this is a work in progress, feel free to add to it -- the list of methods & properties that I know about is at the bottom. My intention for this article is to be a quick-reference guide; all the information you might need about a method or property condensed into a few lines. This might not be possible for something like __slots__, perhaps we should link to the Python Docs for details on the more complex ones. I know this stuff is documented in other places (for example, http://docs.python.org/ref/specialnames.html), but it's all over the place and hard to mentally parse. I want to teach while at the same time to be brief, if that is possible. Anyways feel free to add, don't worry about messing stuff up, I can always revert your edits if I really hate them ; ) And if anyone can figure out where this parsing error is coming from that would be great...

Intro

Methods

For consistancy's sake, let's say we have a class called 'Class', instances called 'x' and 'y'. Keeping with the Python docstrings, '<==>' can be read as 'is equivalent to'.

Creation, Calling, and Destruction

__init__(self, ...)
Class.__init__(...) <==> x.__init__(...) <==> Class(...)
Called when class is called, or in other words when you instantiate the class. Shouldn't return any value; the created instance will be returned automatically.
If not present, nothing is called and life goes on.

__call__(self, ...)
x.__call__(...) <==> x(...)
Called when instance is called.
If not present, instance is not callable and a TypeError (object is not callable) or an AttributeError is raised. (?)

__del__(self)
Called when instance is about to be destroyed. This happens when the reference count reaches zero, probably because the instance was del'ed.
If not present, nothing is called and life goes on.

Conversion to Strings

__repr__(self)
x.__repr__() <==> repr(x) <==> `x`
Should return a string representation of the class or instance with as much information as possible, preferably something that can be passed to eval to recreate the object. Return value must be a string.
If not present, returns something like '<class __main__.Class at 0x2e6d50>' for classes and '<__main__.Class instance at 0xbfb70>' for instances.

__str__(self)
x.__str__() <==> str(x) <==> print x
Should return an informal, user-friendly string describing the class or instance. Return value must be a string.
If not present, __repr__ is tried. If that is not present, returns something like '__main__.Class' for classes and '<__main__.Class instance at 0xbfb70>' for instances.

__unicode__(self)
x.__unicode__() <==> unicode(x)
Should return an informal, user-friendly unicode string describing the class or instance. Return value must be unicode string, or a string or buffer (which will be automatically converted into a unicode string).
If not present, __str__ is called and the result is converted into a unicode string.

Truth Testing

__nonzero__(self)
bool(x) <==> x.__nonzero__()
Called by the built-in function bool, or whenever any truth-value test occurs.
Should return False, True, 0, or 1.
If not defined, __len__ is called; if both are not defined the instance always evaluates to True.

Comparisons

Note: There are no explicit 'reflected' (swapped-argument) versions of the comparison operators.

__cmp__(self, other)
x.__cmp__(other) <==> cmp(x, other)
Should return a negative integer if instance is less than other, 0 if instance is equal to other, or a positive integer if instance is greater than other.
If not present (and other comparisons aren't present?) instances are compared by object identity ('address') (?)

"Rich Comparison" Operators

The following functions customarily return True or False, but can return any value. If in a boolean context, Python will call bool() on the returned value to determine truthfulness. NotImplemented should be raised if no valid comparison can be made.

If not defined, __cmp__ will be called. Python does not attempt to call, say, __lt__ and __eq__ if __le__ is not defined, but instead just calls __cmp__.

__eq__(self, other)
x.__eq__(other) <==> x == other
Should return a value indicating if instance is equal to other.

__ne__(self, other)
x.__ne__(other) <==> x != other
Should return a value indicating if instance is not equal to other.

__lt__(self, other)
x.__lt__(other) <==> x < other
Should return a value indicating if instance is less than other.

__le__(self, other)
x.__le__(other) <==> x <= other
Should return a value indicating if instance is less than or equal to other.

__ge__(self, other)
x.__ge__(other) <==> x >= other
Should return a value indicating if instance is greater than or equal to other.

__gt__(self, other)
x.__gt__(other) <==> x > other
Should return a value indicating if instance is greater than other.

Logical and Mathematical Operators

Regular Binary Operations

These methods implment the binary operators +, -, , /, //, %, *, <<, >>, &, |, and ^, and the functions divmod and pow.

The method may raise NotImplemented if no operation is possible with the supplied with the supplied argument(s).

If not defined or if returns NotImplemented, and the two instances belong to different classes, Python will try to call the equivalent reflected binary operator method of the other instance, with this instance as the argument.

If that method is also not defined, the relevant operation is not possible and a TypeError ('unsupported operand type') is raised.

__add__(self, other)
x.__add__(other) <==> x + other
If not defined or returns NotImplemented, other.__radd__(x) is tried.

__sub__(self, other)
x.__sub__(other) <==> x - other
If not defined or returns NotImplemented, other.__rsub__(x) is tried.

__mul__(self, other)
x.__mul__(other) <==> x * other
If not defined or returns NotImplemented`, ``other.__rmul__(x) is tried.

__div__(self, other)
x.__div__(other) <==> x / other
If from __future__ import division is used, __truediv__ is called instead.
If not defined or returns NotImplemented, other.__rdiv__(x) is tried.

__truediv__(self, other)
x.__truediv__(other) <==> x / other
Only called if from __future__ import division is used. Otherwise, ``__div__` is used.
If not defined or returns NotImplemented, other.__rtruediv__(x) is tried.

__floordiv__(self, other)
x.__floordiv__(other) <==> x // other
If not defined or returns NotImplemented, other.__rfloordiv__(x) is tried.

__mod__(self, other)
x.__mod__(other) <==> x % other
If not defined or returns NotImplemented, other.__rmod__(x) is tried.

__divmod__(self, other)
x.__divmod__(other) <==> divmod(x, other)
Should be equivelant to using __floordiv__ and __mod__. Should not be related to __truediv__.
If not defined or returns NotImplemented, other.__rdivmod__(x) is tried.

__pow__(self, other [, mod])
x.__pow__(other [, mod]) <==> x ** other [% mod] <==> pow(x, other [, mod])
If no mod argument is accepted, the ternary version of pow cannot be used.
If not defined or returns NotImplemented and mod argument is not given, other.__rpow__(x) is tried.
If not defined or returns NotImplemented and mod argument is given, no other methods will be tried and a TypeError (?) is raised.

__lshift__(self, other)
x.__lshift__(other) <==> x << other
If not defined or returns NotImplemented, other.__rlshift__(x) is tried.

__rshift__(self, other)
x.__rshift__(other) <==> x >> other
If not defined or returns NotImplemented, other.__rrshift__(x) is tried.

__and__(self, other)
x.__and__(other) <==> x & other
If not defined or returns NotImplemented, other.__rand__(x) is tried.

__or__(self, other)
x.__or__(other) <==> x | other
If not defined or returns NotImplemented, other.__ror__(x) is tried.

__xor__(self, other)
x.__xor__(other) <==> x ^ other
If not defined or returns NotImplemented, other.__rxor__(x) is tried.

System Message: WARNING/2 (<string>, line 125); backlink

Inline literal start-string without end-string.

Reversed Binary Operations

These methods implement, with swapped commands, the binary operators +, -, , /, //, %, *, <<, >>, &, |, and ^, and the functions divmod and pow. They are only called if the target's equivalent regular binary operator is not defined or returns NotImplemented.

In other words, if y + x is executed, and y.__add__ is not defined, and y and x are of different types, Python will attempt to call x.__radd__(y).

The method may raise NotImplemented if no operation is possible with the supplied with the supplied argument(s).

If not defined, the relevant operation is not possible and a TypeError ('unsupported operand type') is raised.

__radd__(self, other)
x.__radd__(other) <==> other + x
Only called if other.__add__ is not defined or other.__add__(x) returns NotImplemented.

__rsub__(self, other)
x.__rsub__(other) <==> other - x
Only called if other.__sub__ is not defined or other.__sub__(x) returns NotImplemented.

__rmul__(self, other)
x.__rmul__(other) <==> other * x
Only called if other.__mul__ is not defined or other.__mul__(x) returns NotImplemented.

__rdiv__(self, other)
x.__rdiv__(other) <==> other / x
Only called if other.__div__ is not defined or other.__div__(x) returns NotImplemented.
If from __future__ import division is used, __rtruediv__ is called instead.

__rtruediv__(self, other)
x.__rtruediv__(other) <==> other / x
Only called if other.__truediv__ is not defined or other.__truediv__(x) returns NotImplemented.
Also only called if from __future__ import division is used. Otherwise, __rdiv__ is called.

__rfloordiv__(self, other)
x.__rfloordiv__(other) <==> other // x
Only called if other.__floordiv__ is not defined or other.__floordiv__(x) returns NotImplemented.

__rmod__(self, other)
x.__mod__(other) <==> other % x
Only called if other.__mod__ is not defined or other.__mod__(x) returns NotImplemented.

__rdivmod__(self, other)
x.__rdivmod__(other) <==> divmod(other, x)
Should be equivelant to using __rfloordiv__ and __rmod__. Should not be related to __rtruediv__.
Only called if other.__divmod__ is not defined or other.__divmod__(x) returns NotImplemented.

__rpow__(self, other)
x.__rpow__(other) <==> other ** x <==> pow(x, other)
Ternary pow will not try to call __rpow__, as coersion rules would be too complicated. So, __rpow__ only needs to support binary operation.
Only called if other.__pow__ is not defined or other.__pow__(x) returns NotImplemented.

__rlshift__(self, other)
x.__rlshift__(other) <==> other << x
Only called if other.__lshift__ is not defined or other.__lshift__(x) returns NotImplemented.

__rrshift__(self, other)
x.__rrshift__(other) <==> other >> x
Only called if other.__rshift__ is not defined or other.__rshift__(x) returns NotImplemented..

__rand__(self, other)
x.__rand__(other) <==> other & x
Only called if other.__and__ is not defined or other.__and__(x) returns NotImplemented.

__ror__(self, other)
x.__ror__(other) <==> other | x
Only called if other.__or__ is not defined or other.__or__(x) returns NotImplemented.

__rxor__(self, other)
x.__rxor__(other) <==> other ^ x
Only called if other.__xor__ is not defined or other.__xor__(x) returns NotImplemented.

In-Place Binary Operations

These methods implement the in-place binary operators +=, -=, *=, /=, //=, %=, **=, <<=, >>=, &=, |=, and ^=. Note that there is no in-place equivalent for divmod, while pow is only available as **=.

The method should attempt to modify the instance (self) in place and return the result. In any case, Python assigns the result as the new value of the instance.

It may raise NotImplemented if no operation is possible with the supplied with the supplied argument(s).

If method is not defined or returns NotImplemented, the equivalent operation is attempted with the the equivalent regular binary operator. If that is not defined or returns NotImplemented, and the two instances are of different types, the equivalent reflected binary operator method of the other instance is called.

If none of the three is defined or don't raise Not Implemented, the operation is not possible and a TypeError ('unsupported operand type') is raised.

For example, if x += other is attempted, x.__iadd__(other) is tried first, followed by x.__add__(other) and then other.__radd__(x).

__iadd__(self, other)
x.__iadd__(other) <==> x += other
If not defined or returns NotImplemented, x.__add__(other) is tried, followed by other.__radd__(x).

__isub__(self, other)
x.__isub__(other) <==> x -= other
If not defined or returns NotImplemented, x.__sub__(other) is tried, followed by other.__rsub__(x).

__imul__(self, other)
x.__imul__(other) <==> x *= other
If not defined or returns NotImplemented, x.__mul__(other) is tried, followed by other.__rmul__(x).

__idiv__(self, other)
x.__idiv__(other) <==> x /= other
If from __future__ import division is used, __itruediv__ is called instead.
If not defined or returns NotImplemented, x.__div__(other) is tried, followed by other.__rdiv__(x).

__itruediv__(self, other)
x.__itruediv__(other) <==> x /= other
Only called if from __future__ import division is used. Otherwise, __idiv__ is used.
If not defined or returns NotImplemented, x.__truediv__(other) is tried, followed by other.__rtruediv__(x).

__ifloordiv__(self, other)
x.__ifloordiv__(other) <==> x //= other
If not defined or returns NotImplemented, x.__floordiv__(other) is tried, followed by other.__rfloordiv__(x).

__imod__(self, other)
x.__imod__(other) <==> x %= other
If not defined or returns NotImplemented, x.__mod__(other) is tried, followed by other.__rmod__(x).

__ipow__(self, other)
x.__ipow__(other) <==> x **= other
If not defined or returns NotImplemented, x.__pow__(other) is tried, followed by other.__rpow__(x).

__ilshift__(self, other)
x.__ilshift__(other) <==> x <<= other
If not defined or returns NotImplemented, x.__lshift__(other) is tried, followed by other.__rlshift__(x).

__irshift__(self, other)
x.__irshift__(other) <==> x >>= other
If not defined or returns NotImplemented, x.__rshift__(other) is tried, followed by other.__rrshift__(x).

__iand__(self, other)
x.__iand__(other) <==> x &= other
If not defined or returns NotImplemented, x.__and__(other) is tried, followed by other.__radd__(x).

__ior__(self, other)
x.__ior__(other) <==> x |= other
If not defined or returns NotImplemented, x.__or__(other) is tried, followed by other.__ror__(x).

__ixor__(self, other)
x.__ixor__(other) <==> x ^= other
If not defined or returns NotImplemented, x.__xor__(other) is tried, followed by other.__rxor__(x).

Unary Operations

__neg__(self)
x.__neg__() <==> -x
If not defined, this construct cannot be used and an AttributeError is raised.

__neg__(self)
x.__pos__() <==> +x
If not defined, this construct cannot be used and an AttributeError is raised.

__invert__(self)
x.__invert__() <==> ~x
If not defined, this construct cannot be used and an AttributeError is raised.

__abs__(self)
x.__abs__() <==> abs(x)
If not defined, the abs function cannot be used on this instance and an AttributeError is raised.

Casts

These methods implement conversion to various numeric types. One of these methods is called when the corresponding built-in method is called with the instance. If no conversion is possible, a ValueError can be raised (?) (not sure if this is official, but that's what strings do). If not present, the specified conversion cannot be done and an AttributeError is raised.

Real Numbers

In these methods, if the incorrect type is returned, a TypeError is raised (e.g., "__float__ returned non-float").

__float__(self)
x.__float__() <==> float(x)
Should return a floating-point representation of x, as a float instance.

__hex__(self)
x.__hex__() <==> hex(x)
Should return a hexadecimal representation of x, preceded by "0x", as a string.

__int__(self)
x.__int__() <==> int(x)
Should return a integer representation of x, as an int instance.
Note: Although the built-in int accepts a base value, this appears to be only useable if a class inherits from str (?) and appears not to involve str.__int__ (?).

__long__(self)
x.__long__() <==> long(x)
Should return a long-integer representation of x, as a long instance.
Note: Although the built-in long accepts a base value, this appears to be only useable if a class inherits from str (?) and appears not to involve str.__long__ (?).

__oct__(self)
x.__oct__() <==> oct(x)
Should return a octal representation of x, preceded by "0", as a string.

Complex Numbers

__complex__(self)
x.__complex__() <==> complex(x)
x.__complex__() + y.__float__() <==> complex(x,y)
Should return a complex number. If another type is returned, Python will attempt to call its __float__ method and assign the result to the real part of the complex number.
If a second argument (the imaginary part) is given to the built-in complex method, Python will call its __float__ method and add the result to the imaginary part of the complex number.

Containing Items

Basics

Although not required, it's probably a good idea to implement these if your instances will contain user-accessible items.

__len__(self)
x.__len__() <==> len(x)
Should (must?) return an integer describing how may items are contained by the instance.
If __nonzero__ is not defined and __len__ returns 0, instance is considered to evaluate to False. If neither is present instance will always evaluate to True.
If not present, len cannot be used and negative slice indexes cannot be used for __getslice__, __setslice__, and __delslice__. An AttributeError is raised.

__contains__(self, item)
x.__contains__(item) <==> item in x
Should return True if item is contained by the instance, False otherwise. Mapping objects should consider only keys, not values.
If not present, Python attempts to look for the item by iterating over all items using __iter__ and then __getitem__, with integer keys starting at zero and ending when IndexError is raised. If neither is defined, a TypeError (?) is raised.

__iter__(self)
x.__iter__() <==> iter(x)
Also called when iteration over the instance is requested, such as in a for loop.
Should return an iterator suitable for iterating over all the items contained in the instance. Mapping objects, again, should consider only keys.
If not present, Python attempts to call __getitem__ with integer keys starting at zero and ending when IndexError is raised. If __getitem__ is also not defined, a TypeError ('iteration over non-sequence') is raised.

Items and Slices

These methods are called when bracket notation is used.

Python will behave differently depending on the type of value inside of the brackets:

x[key], where key is a single value
Calls x.__*item__(key)
x[start:end] where x.__*slice__ exists
Calls x.__*slice__(cooked_start, cooked_end) where start and end are 'cooked' as described below in 'Old-Style Slices'
x[start:end] where x.__*slice__ does not exist, or x[extended_slice], where extended slice is any slice more complex than start:end
Calls x.__*item__ with slice object, Ellipsis, or list of these.
Items and New-Style Slices

In general, if key is of an inappropriate type, TypeError should be raised. If it is outside the sequence of keys in instance, IndexError should be raised. If instance is a mapping object and key cannot be found, KeyError should be raised. (What if neither of these is true? I don't know.)

__getitem__(self, key)
x.__getitem__(key) <==> x[key]
Should return item(s) referenced by key.
Not called if __setslice__ exists and simple start:end slicing is used.
If not present, items cannot be evaluated using bracket notation, and an AttributeError is raised.

__setitem__(self, key, value)
x.__setitem__(key, value) <==> x[key] = value
Should set or replace item(s) referenced by key. value can be a single value or a sequence.
Not called if __setslice__ exists and simple start:end slicing is used. Usage not dependent on presence of __getitem__.
If not present, items cannot be assigned using bracket notation, and an AttributeError is raised.

__delitem__(self, key)
x.__delitem__(key) <==> del x[key]
Should delete item(s) represented by key. Not dependent on presence of __getitem__.
Not called if __delslice__ exists and simple start:end slicing is used. Usage not dependent on presence of __getitem__.
If not present, items cannot be deleted using bracket notation, and an AttributeError is raised.
Old-Style Slices

These methods are depreciated but still widely used. Furthermore, for simple slicing Python checks for their existence first before calling the __*item__ methods.

For these methods, Python 'cooks' the indexes before passing them. Negative slice indexes are converted to (usually) positive ones by adding them to the value returned by __len__, and so if __len__ is not defined, negative slice indexes cannot be used. Furthermore, an empty start index is replaced by 0, and an empty end index by sys.maxint.

__getslice__(self, start, end)
x.__getslice__(start, end) <==> x[start:end]
Should return items in the slice represented by start:end.
If not present, or if extended slicing is used, a slice object is passed to __getitem__.

__setslice__(self, start, end, sequence)
x.__setitem__(start, end, sequence) <==> x[start:end] = sequence
Should assign sequence to the slice represented by start:end.
If not present, or if extended slicing is used, a slice object is passed to __setitem__.

__delslice__(self, key)
x.__delslice__(start, end) <==> del x[start:end]
Should delete items in the slice represented by start:end.
If not present, or if extended slicing is used, a slice object is passed to __delitem__.

Attribute Access

As an alternative to Containing Items, instance attributes can be directly read, written to, and deleted. Access can be customized using these methods.

__getattr__(self, attr)
x.attr <==> x.__getattr__(self, attr)
Only called when attr isn't found in any of the usual places (it is not an instance attribute or a class attribute, and class is old-style or __getattribute__ is not defined).
Should return the value of the key attr in the relevant mapping object.
If not implemented, the attribute cannot be found and an AttributeError is raised.

__getattribute__(self, attr)
x.attr <==> x.__getattribute__(self, attr)
New-style classes only.
Called whenever attribute assignment is attempted (unlike __getattr__).
Should return the value of the key attr in the relevant mapping object.
If not implemented, Python will find the attribute as normal (including calling __getattr__).
If raises AttributeError, Python will call __getattr__, but will not look anywhere else for the attribute.

__setattr__(self, attr, value)
x.attr = value <==> x.__setattr__(self, attr, value)
Called whenever attribute assignment is attempted.
Should give attr a value of value in the relevant mapping object.
If not implemented, Python will assign the attribute to the instance as normal: self.__dict__[attr] = value for old-style classes and object.__setattr__(self, attr, value) for new-style classes.

__delattr__(self, attr)
del x.attr <==> x.__delattr__(self, attr)
Called whenever attribute deletion is attempted.
Should delete the key attr in the relevant mapping object.
If not implemented, Python deletes the attribute as normal: del x.__dict__[attr] for old-style classes and object.__delattr__(self, attr) for new-style classes. (I think, can anyone verify?)

Being Contained By Another Object

__hash__(self)
hash(x) <==> x.__hash__()
Called by the built-in function hash, and to compare keys in dictionary operations.
Should return a 32-bit integer. Objects which evaluate as equal should have the same hash value.
Starting in Python 2.5, may instead return a long integer; the result of its __hash__ will be used.
Only should be defined if __cmp__ is defined; if __cmp__ or __eq__ are defined and __hash__ is not, object cannot be used as a dictionary key. If none of the three are defined, object can be used as a dictionary key (why?).
Hash value should be immutable (always?).

Descriptors

  • __get__
  • __set__
  • __delete__
__index__ x[y:z] <==> x[y.__index__():z.__index__()]

Intro

  • dir()
  • object
  • property/fget/fset/fdel

Misc

__new__ __reduce__ (for pickling) __reduce_ex__ __getnewargs__ __reversed__

__enter__ __exit__ __subclasses__ __coerce__

Class & Object Properties

__dict__ & vars() __class__ __metaclass__ __bases__ __name__ __slots__ __weakref__

Class & Function Properties

__doc__

File Properties

__file__

Builtins

__import__

Module Properties

__builtins__ (nonstandard, see http://docs.python.org/lib/module-builtin.html)

Modules

__builtin__ __main__ (represents scope of main program, http://docs.python.org/lib/module-main.html) __future__

Comments

over 11 years ago (03 Jul 2008 at 01:09 AM) by harry_hill
FAIL on 1st line!

Class.__init__(...) <==> x.__init__(...)
- NO!!! Calling a method on an instance passes self as an implicit first parameter.
over 11 years ago (03 Jul 2008 at 01:10 AM) by harry_hill
x.__init__(...) <!=> Class(...)
- WHAT!? NO!!! Object instantiation implies a call to __new__ to do resource allocation. __init__ only sets initial state!

Why did you write this if you have NO idea what you're talking about?
over 11 years ago (03 Jul 2008 at 01:32 AM) by David Isaacson
That's only for new-style classes actually. I haven't written about __new__ yet, I'll add that part later. Or feel free to.
over 11 years ago (03 Jul 2008 at 01:12 AM) by harry_hill
Still in first section(!!!):
"If not present, nothing is called and life goes on."

NOOOO!!!!!
If not present, an implicit call to the superclass's __init__ method is made.

I didn't get beyond the first paragraph. I hope it gets better...
over 11 years ago (03 Jul 2008 at 01:30 AM) by Stou S.
You know you can just edit the thing...
over 11 years ago (03 Jul 2008 at 01:33 AM) by David Isaacson
That's actually true... but that's assuming you have a superclass. Perhaps I should have been clearer, but when I say 'if not present' I mean 'if not present anywhere in the class hierarchy'