Add potatoes to a large sauté pan and cover with 2 inches of cold water. Place pan over medium-high heat and bring to a boil. Season with 1 teaspoon salt, reduce heat to medium and simmer until potatoes are fork-tender, about 20 minutes. Drain potatoes and transfer potatoes to a bowl to cool slightly.
Whisk together ½ cup sour cream, ¼ cup each chives, dill, and parsley, ½ cup celery, 3 tablespoons capers, 1 tablespoon Dijon, and ½ teaspoon each kosher salt and black pepper.
Halve smaller potatoes and quarter larger potatoes; add to bowl with dressing and toss to coat. Season with additional salt, pepper and caper brine to taste.
Herbs: I recommend using a mixture of fresh herbs—use whatever you like and have on hand. Chives, dill and parsley work well together but tarragon, cilantro, and scallions would all be delicious as well.Storage: store potato salad in a glass container with a tight fitting lid for up to 1 week in the refrigerator. Do not freeze this potato salad.