sparse-checkout: add basics of 'clean' command

sparse-checkout: add basics of 'clean' command

When users change their sparse-checkout definitions to add new
directories and remove old ones, there may be a few reasons why
directories no longer in scope remain (ignored or excluded files still
exist, Windows handles are still open, etc.). When these files still
exist, the sparse index feature notices that a tracked, but sparse,
directory still exists on disk and thus the index expands. This causes a
performance hit _and_ the advice printed isn't very helpful. Using 'git
clean' isn't enough (generally '-dfx' may be needed) but also this may
not be sufficient.

Add a new subcommand to 'git sparse-checkout' that removes these
tracked-but-sparse directories.

The implementation details provide a clear definition of what is happening,
but it is difficult to describe this without including the internal
implementation details. The core operation converts the index to a sparse
index (in memory if not already on disk) and then deletes any directories in
the worktree that correspond with a sparse directory entry in that sparse
index.

In the most common case, this means that a file will be removed if it is
contained within a directory that is both tracked and outside of the
sparse-checkout definition. However, there can be exceptions depending on
the current state of the index:

 * If the worktree has a modification to a tracked, sparse file, then that
   file's parent directories will be expanded instead of represented as
   sparse directories. Siblings of those parent directories may be
   considered sparse.

 * If the user staged a sparse file with "git add --sparse", then that file
   loses the SKIP_WORKTREE bit until the sparse-checkout is reapplied. Until
   then, that file's parent directories are not represented as sparse
   directory entries and thus will not be removed. Siblings of those parent
   directories may be considered sparse. (There may be other reasons why
   the SKIP_WORKTREE bit was removed for a file and this impact on the
   sparse directories will apply to those as well.)

 * If the user has a merge conflict outside of the sparse-checkout
   definition, then those conflict entries prevent the parent directories
   from being represented as sparse directory entries and thus are not
   removed.

 * The cases above present reasons why certain _file conditions_ will impact
   which _directories_ are considered sparse. The list of tracked
   directories that are outside of the sparse-checkout definition but not
   represented as a sparse directory further reduces the list of files that
   will be removed.

For these complicated reasons, the documentation details a potential list of
files that will be "considered for removal" instead of defining the list
concretely. The special cases can be handled by resolving conflicts,
committing staged changes, and running 'git sparse-checkout reapply' to
update the SKIP_WORKTREE bits as expected by the sparse-checkout definition.

It is important to make clear that this operation will remove ignored and
excluded files which would normally be ignored even by 'git clean -f' unless
the '-x' or '-X' option is provided. This is the most extreme method for
doing this, but it works when the sparse-checkout is in cone mode and is
expected to rescope based on directories, not files.

The current implementation always deletes these sparse directories
without warning. This is unacceptable for a released version, but those
features will be added in changes coming immediately after this one.

Note that this will not remove an untracked directory (or any of its
contents) if its parent is a tracked directory within the sparse-checkout
definition. This is required to prevent removing data created by tools that
perform caching operations for editors or build tools.

Thus, 'git sparse-checkout clean' is both more aggressive and more careful
than 'git clean -fx':

 * It is more aggressive because it will remove _tracked_ files within the
   sparse directories.

 * It is less aggressive because it will leave _untracked_ files that are
   not contained in sparse directories.

These special cases will be handled more explicitly in a future change that
expands tests for the 'git sparse-checkout clean' command. We handle some of
the modified, staged, and committed states including some impact on 'git
status' after cleaning.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>