It would work - and while your current solution works fine for your specific purpose, what if you later on decide you want to test against a different picture box, or that you want to add more than one collider (for example, you could create a simple game where the player must avoid several obstacles)?
The difference is that, in your case, since you're relying on the Visible property, you don't have to remove the picture box from the Control collection - you can just set Visible to false; this way it's easier to switch it on and off at will.

That said, there is a slight problem in your code above - you're lucky it works, because, it just so happens that one of the calls you make on the picBox1 has a convenient side effect of setting the Visible property to false. Note that, just by reading your code you can't tell why would the if condition fail on the next test. The method call that makes it all work is the call to the Dispose() method. However, once a control is disposed, it cannot be used normally anymore - some method and property calls will fail, others will not, and once again, you were lucky that the Visible property is still works.
As all this behavior is not documented (you can't find anything on MSDN that explicitly states that calling Dispose sets Visible to false, or that Visible is guaranteed not to throw an ObjectDisposedException), in theory it could change from one implementation of .NET Library to another, so you shouldn't rely on it.

It's better that you add one more line of code where you explicitly set the Visible property to false.

Code:
if (picBox1.Visible)
{
    if (picBoxBal.Bounds.IntersectsWith(picBox1.Bounds))
    {
        BalSpeedY = -BalSpeedY;
        score++;
        LblScore.Text = "Score: " + score.ToString();
        picBox1.Visible = false;        

        // Now you can safely ditch these, which leaves you with the option to turn it back on in a simple way:
        // this.Controls.Remove(picBox1);
        // picBox1.Dispose();
    }
}

This way, everything is much more clear, and it will work for sure.
Some other things you can do; you can extract this code into a method, and pass the collider to test against as a parameter:

Code:
private bool TestForCollisionWith(Control target)
{
    bool result = false;

    if (target.Visible)
    {
        if (picBoxBal.Bounds.IntersectsWith(target.Bounds))
        {
            BalSpeedY = -BalSpeedY;
            score++;
            LblScore.Text = "Score: " + score.ToString();
            target.Visible = false;

            result = true;
        }
    }

    return result;   // false if no collision, true if a collision was detected
}


// And then you can call this method from where the original code used to be:
TestForCollisionWith(picBox1);    // BENEFIT: you can now pass something other than picBox1
The method returns a bool indicating if a collision was detected or not, for convenience.

Finally, you can use a List<Control> and a loop in the same way as arrokai, except now the only thing you'd need to do inside the loop is to call the TestForCollisionWith() method. You can also remove the picture box from the list (but you don't have to, as now the code tests the Visible property; not removing it introduces some redundancy though, but for a small number of stored objects this is not a big problem).