When making a selection in Photoshop, accuracy is everything. There are several tools built in to Photoshop that can help you make a selection and create a mask, but not all of them will smooth out the edges.
In this tutorial, we're going to get Photoshop to detect our subject, and then use a couple of tools to learn how to soften the edges of a cutout in Photoshop.
What You'll Learn in This Tutorial on Using Photoshop to Smooth Out Edges
- How to make an accurate selection
- How to clean up edges in Photoshop
What You'll Need for this "How to Smooth Edges in Photoshop" Tutorial
Our objective will be to seamlessly cut out our subject from the environment, clean up the edges in Photoshop, and place them onto a white background. Let's use this portrait of a woman with a camera from Envato Elements by ssvetasokolova.
In case you don't already know, Envato Elements is an amazing resource for all kinds of stock images, stock video, design templates, and so much more. Be sure to check it out for any of your design-related needs.
1. How to Make a Selection in Photoshop
Let's start this tutorial on how to smooth jagged edges in Photoshop. Open the portrait in Photoshop. Go to Select > Subject to use the powerful built-in AI in Photoshop to create a selection of only the subject.
You should now see 'marching ants' around our subject. This is showing the selection of what Photoshop has detected as the subject of the image.
A lot of people would see this selection and be satisfied. However, if we stopped here, the edges of our subject would not be at all smooth as we still need to smooth out the rough edges. Let's place this selection on a white background so we can easily see the selection and where the edges are jagged.
Notice, especially along the edges of this woman's jacket, how we have some nasty jagged edges? Here's a detail on the bottom right of her sweater sleeve to show these edges.
This is not ideal, and our selection needs improvement. Let's continue and learn how to refine this original 'Select Subject' selection and remove the rough edges in Photoshop.
2. How to Clean Up Edges in Photoshop
You should still be seeing marching ants on your subject at this point. If you've deselected your selection, go back and Select > Subject to bring back this selection.
My favourite way to smooth jagged edges in Photoshop is by using a combination of the Contract and Feather tools. First, let's contract our selection. Go to Select > Modify > Contract to open this tool.
The contract tool will make our selection smaller, by however many pixels we choose. We want to remove as little as we can from our selection, so let's keep the contract adjustment very small. I've chosen to Contract by 2 pixels. Leave 'Apply effect at canvas bounds' unchecked.
Now that our selection is slightly smaller, the last step is to soften our selection. This is done by using the Feather tool. Go to Select > Modify > Feather to open this tool.
Because the details in the sleeves are quite intricate, we'll need a decent amount of feathering to softly blend those details into our selection. I've chosen a Feather of 6 pixels. Leave 'Apply effect at canvas bounds' unchecked.
Just like that, we've refined our selection and smoothed out the rough edges. All that's left to do is to place our selection on a white background.
Go to Select > Inverse so our selection switches from our subject to everything except our subject.
Now, with our background selected, go to Edit > Fill.
Select White from the Contents menu.
And look at these results! All of a sudden, those pixelated edges of our subject have been smoothed out.
Let's zoom in on that same bottom right sleeve from earlier to see the before and after of our refined selection.
Awesome Work, You're Done!
We've now successfully used Photoshop to remove pixelated edges from our selection, and created a seamless edit!
This technique works on just about anything, from a portrait to a still life, graphics, and just about anything you may need to use Photoshop to remove pixelated edges. I hope you'll give it a try and see how this technique can work for you the next time you need to smoothen out edges in Photoshop.