The brute force you are talking about has asymptotic complexity O( n^3 ). You can go down to O( n^2 * lg(n) ) if you do the following...
1. For each point A precompute the angel of the line AB for each point B - O( n^2 )
2. Sort the lines of the just generated matrix - O( n^2 * lg(n) )
3. For each line AB binary search through the list of precomputed angles of point A for the desired angle - O( n^2 * lg(n) )
Be careful for roundings.
Sorry, that's the best I can come up with.
That's still O( n^2 * lg(n) ) if implemented correctly, but definitely easier to be written than mine.
There are also no rounding problems, because you never need to calculate the square root (having the squared distance between two points is actually what you really need).
Thanks all for the replies. Muthuveerappan, yes that's the method I will use if there isn't other.
yzakov, I can't clearly understand what u mean by O(n^2*lg(n)). Could you please PM me with the message on Bulgarian? Thanks again for the help.