public/git-how.md
... ...
@@ -1 +1,443 @@
1
-hold
... ...
\ No newline at end of file
0
+Best resources
1
+
2
+* How it really works
3
+ * <https://neoteric.eu/do-the-magic-with-git>
4
+* Implementations
5
+ * <https://maryrosecook.com/blog/post/git-from-the-inside-out>
6
+ * <https://robots.thoughtbot.com/rebuilding-git-in-ruby>
7
+* Background
8
+ * <https://en.wikipedia.org/wiki/Merkle_tree>
9
+* Navigating
10
+ * <http://correl.phoenixinquis.net/2015/07/12/git-graphs.html>
11
+
12
+We are going to make a repo for a wiki. Along the way we are going to watch what Git is doing.
13
+
14
+### Why is Git so hard?
15
+
16
+In my opinion, the reason Git is so hard is that the commands (`git add`, `git commit`, `git checkout`, etc.),
17
+have names whose semantics seem to be mapped onto the history of a variety of versioning tools, especially Subversion. The actual git commands, however, have behaviors with very precise meanings regarding specific git entities. For instance, `git add` means: Compress the files and add them to a git-managed directory, and add references to those files to the binary "index" file. Even when the official Git docs use the language of the implementation (for instance, we are told by `git help add` that `git-add` means "Add file contents to the index") it isn't very clear what
18
+"index" means in this context. A lot would be gained by banning words such as "staging area" and "cache" and just sticking with "index."
19
+
20
+### To get started
21
+
22
+Create a couple of extra terminal tabs, and in the second tab,
23
+
24
+ watch-do $HOME/src/wiki $HOME/src/wiki "tree -t -r -C -a ."
25
+
26
+This lets us observe the directory hierarchy Git creates to manage the state of our repository. [To set up `watch-do` see the end of this document.]
27
+
28
+### Create the wiki
29
+
30
+In the first tab,
31
+
32
+ cd $HOME
33
+ cd src
34
+ rm -fR wiki
35
+ mkdir wiki
36
+ cd wiki
37
+
38
+### Create a file and initialize the repository
39
+
40
+ echo "### Welcome to the wiki" >home.md
41
+ git init
42
+
43
+What are these directories and files?
44
+
45
+`home.md` (and everything else except for `.git/`) is your content in what Git calls the "working directory."
46
+
47
+The `.git/` directory holds the entire history of the repository. Within that,
48
+
49
+| Name | What |
50
+|-----------------|------------------------------------------------------------------------------------------------------------------------|
51
+| `index` | (You won't see this until something has been added.) A binary file with a sorted list, each item being a hash of a blob object and its permission. The index becomes the basis for the next "tree" object that will be committed. |
52
+| `refs/tags/` | |
53
+| `refs/heads/` | |
54
+| `objects/` | |
55
+| `objects/pack/` | |
56
+| `objects/info/` | |
57
+| `config` | A text file with configuration information for the repository. |
58
+| `HEAD` | An ordinary text file that tracks a reference to whatever branch is the current head. |
59
+| `description` | (You may not see this) If present, contains a name for the repository. Used by the GitWeb program. |
60
+| `hooks/` | (You may not see this) Contains files that define behaviors that are triggered at various times during Git operations. |
61
+| `info/` | (You may not see this) Contains a global exclude file for patterns you don’t want to track in a .gitignore file |
62
+
63
+### Add a file and watch the index
64
+
65
+ git add home.md
66
+
67
+In a third tab:
68
+
69
+ watch-do $HOME/src/wiki $HOME/src/wiki/.git/index "git ls-files --stage"
70
+
71
+The concept: We are adding `home.md` to the "index." The index is a place where changes accumulate
72
+that you will eventually commit to your repo. **Think of the index, then, as "the next commit."** (Ryan Tomayko
73
+has a nice post on this, though he calls the index "the next patch" -- see <http://2ndscale.com/rtomayko/2008/the-thing-about-git>.) This is the place where you set up your changes properly before making them official.
74
+
75
+A directory has been added in `.git/objects/` called `81` and within that, a file
76
+called `d4aaeff611587c378d25a90e1d2f178b484727`. Additionally, `.git/index` has changed. The idea
77
+is that `objects/` is effectively a database of compressed files, and the file names are based
78
+on the hashes, which are based on the contents. By this means, `git` is a content-addressable
79
+database. Notice that the contents of `objects/` has no information regarding filenames; it's
80
+all about content.
81
+
82
+Since `objects/` just contained compressed objects, there needs to be a structure
83
+that manages the current state: That's what the `index` is for: A list of hashes pointing into
84
+the `objects/` store, with the names used when you added.
85
+
86
+Indeed, if you added a second identical file with a different name with
87
+`echo "### Welcome to the wiki" >home2.md` you will find that what's in `objects/` doesn't change.
88
+If you then add that file with `git add home2.md` the only change you will find is in the index,
89
+which will reference the name object, but with a different name.
90
+
91
+Again, the exact sequence:
92
+
93
+1. A directory `81/` is created. Within `81/` a file `d4aaeff611587c378d25a90e1d2f178b484727`
94
+ is created. [The path `81/d4aaeff611587c378d25a90e1d2f178b484727` is a hash of the content
95
+ of the `home.md` file, which you can get with `git hash-object home.md`
96
+ (try it). `git hash-object` is pretty simple; here's a Ruby version of how to get a git-compatible
97
+ hash for a blob: <https://gist.github.com/jgn/b3dddf091db54b2d719fa02e996dc5a3>
98
+2. The compressed contents of the file `home.md` are put the file `81/d4aaeff611587c378d25a90e1d2f178b484727` --
99
+ to see this, type `git show 81d4aa` (`81d4aa` is the opening part of the hash). Or you can manually
100
+ uncompress it with
101
+
102
+ ruby -rzlib -e 'print Zlib::Inflate.new.inflate(STDIN.read)' < .git/objects/81/d4aaeff611587c378d25a90e1d2f178b484727
103
+
104
+ The git way to see the content (`-p` means "pretty print"):
105
+
106
+ git cat-file -p 81d4aaeff611587c378d25a90e1d2f178b484727
107
+
108
+ And to see the type . . .
109
+
110
+ git cat-file -t 81d4aaeff611587c378d25a90e1d2f178b484727
111
+
112
+3. A binary file `.git/index` is added. An entry is added to `.git/index` with the mode of the file,
113
+ the hash, and the filename. To see this, type `git ls-files --stage` (which is what is being shown
114
+ in your third terminal tab, if you've been following along. Also, `git ls-files --cached` does
115
+ the same thing as `--stage` but just lists the filenames only -- `--cached` is the default.)
116
+
117
+### Make the first commit to the repo
118
+
119
+Commit everything that has been added (everything in the index)
120
+
121
+ git commit -m "Initial commit"
122
+
123
+Observations
124
+
125
+1. The index stays the same
126
+2. There are now a total of three objects; two are new. One is a tree.
127
+
128
+ $ git cat-file -t 8dfcc6
129
+ tree
130
+ $ git cat-file -p 8dfcc6
131
+ 100644 blob 81d4aaeff611587c378d25a90e1d2f178b484727 home.md
132
+
133
+ It contains a pointer to the blob, and also includes a name, so that when the tree is applied, the
134
+ blob will get a name.
135
+
136
+ All of the paths are relative to the root of the project.
137
+
138
+3. The other new object is a commit. Note that the hash will be different from what I'm using, because the hash
139
+ is based on data that is different on your system, namely your name/email and the date/time of the commit,
140
+ which are included in the commit messages and change the hash.
141
+
142
+ $ git cat-file -t 0e12bc
143
+ commit
144
+ $ git cat-file -p 0e12bc
145
+ tree 8dfcc6e23dec849d71d7428290cf66aab644a22f
146
+ author John Norman <john@iorahealth.com> 1475667520 -0500
147
+ committer John Norman <john@iorahealth.com> 1475667520 -0500
148
+
149
+ Initial commit
150
+
151
+ The commit points to the tree. The tree points to the blob.
152
+
153
+### Now add a second file
154
+
155
+In the first tab,
156
+
157
+ echo "Software we like: git, atom" > software.md
158
+ git add software.md
159
+
160
+You should see a new item `ed/91a30546d771a1c22ab2d1e846bdea6b52a624` in the `objects/` directory, and a new line added to the index.
161
+
162
+### Let's check our status
163
+
164
+In the first tab,
165
+
166
+ git status
167
+
168
+Notice that the heading before the files in the index says: "Changes to be committed" and that we are told that the command to "unstage" a file is to type `git rm --cached <file> ...`. Thus already we have a proliferation of terms regarding the files that are compressed in `.git/objects` and listed in `.git/index` -- the terms are (1) changes to be committed; (2) "staged"; (3) "cached." These all mean the same thing. Nowhere is the word "index" mentioned, even though it is in the docs if you type `git help add`. But we know what all this stuff means: `git status` is telling us what is in `.git/objects` and `.git/index`.
169
+
170
+### Let's add a directory and another file and see what happens
171
+
172
+Let's create an `images/` directory and put something in there:
173
+
174
+ mkdir images
175
+ curl -s http://i2.kym-cdn.com/photos/images/original/000/173/575/25810.jpg -o images/wat.jpg
176
+ git add images
177
+
178
+Notice that we have added the compressed image file `d2/dd63f0a1e8f1c07f4c33e5af782a60fda3546f` to `.git/objects` as well as an entry in `.git/index`. The newly added directory (`images/`)
179
+is only indicated as the directory for the `wat.jpg` file (there is no separate addition for the index for `images/` itself. Also, if we
180
+type
181
+
182
+ git rm --cached images/wat.jpg
183
+
184
+the lines is taken out of the index, but the compressed file remains in `objects/`. [**NOTE:** The fact that the compressed file remains in `objects/` provides you some safety should you remove files from the index and from your working copy. But if you have not "pushed" your repo to a remote, deleting stuff from the `objects/` hierarchy can result in loss-of-data. In short, leave `objects/` alone, except for manipulations via git commands.]
185
+
186
+Do this:
187
+
188
+ rm images/wat.jpg
189
+ git add images
190
+
191
+are not files there, and . . . nothing happens to the index. This is because Git cannot represent empty directories in the index (staging area) -- See <https://git.wiki.kernel.org/index.php/GitFaq#Can_I_add_empty_directories.3F>.
192
+
193
+At this point, we removed `images/wat.jpg` from the index AND from our working directory. And yet we can still see the compressed
194
+file in the `.git/objects/` hierarchy under `d2/dd63f0a1e8f1c07f4c33e5af782a60fda3546f` which we can peek at with `git cat-file -p d2dd63f0a1e8f1c07f4c33e5af782a60fda3546f | imgcat`. Thus the arbiter of what Git knows is `.git/index` not the assortment of
195
+things in the `.git/objects` directory. In other words, you may find that you've removed a file from your working directory,
196
+`git status` doesn't know about it, but the file is still safe in the `.git/objects` directory.
197
+
198
+Now, re-do the curl command to get the image,
199
+
200
+ curl -s http://i2.kym-cdn.com/photos/images/original/000/173/575/25810.jpg -o images/wat.jpg
201
+
202
+and `git add images`. What's in `.git/objects` doesn't change (the image was already saved there) but
203
+the index changes.
204
+
205
+As you can see, the fact that `wat.jpg` lives in `images/` is only known to the index. You can get a feel for this
206
+by removing `images/wat.jpg` from the index with `git rm --cached images/wat.jpg`, then adding a new directory `images2`
207
+and then curl'ing a new `wat.jpg`, and then adding `images2/`:
208
+
209
+ git rm --cached images/wat.jpg
210
+ mkdir images2
211
+ curl -s http://i2.kym-cdn.com/photos/images/original/000/173/575/25810.jpg -o images2/wat.jpg
212
+ git add images2
213
+
214
+You will see that only the index changes -- the content
215
+has already been hashed into the objects hierarchy. Reverse the experiment with . . .
216
+
217
+ git rm --cached images2/wat.jpg
218
+ rm -fR images2
219
+ git add images
220
+
221
+Finally, before we commit, let's add one more file to the working directory that will help us understand the relationship
222
+between `git status`, the index, and the last commit.
223
+
224
+ echo -e '### Image list\n\n* more images.md ![](images/wat.jpg)' >images.md
225
+
226
+### Now let's see what happens if we do a commit
227
+
228
+Commit files with
229
+
230
+ git commit -m "Add pages and images"
231
+
232
+You will notice that this immediately creates some new items in `objects/`, a new directory `logs/` with some items, and a new directory `refs/` with some items. What has happened?
233
+
234
+First, the new objects. There are three of them. Use `git cat-file -p` and `git cat-file -t` to inspect them.
235
+
236
+1. `.git/objects/f9/c298b9e62cecfc65ede840f48fa025347909fe` is the **tree** with a blob entry for `wat.jpg`. Doing `git cat-file -p f9c298b9e62cecfc65ede840f48fa025347909fe` we see
237
+
238
+ 100644 blob d2dd63f0a1e8f1c07f4c33e5af782a60fda3546f wat.jpg
239
+
240
+ Do we know the name of the tree? No. Do we know `wat.jpg` is kept in a tree? Yes.
241
+
242
+2. `.git/objects/11/dd91adca91dc2445dfcee4d3418e8ad219db7c` is another **tree** created from the index that
243
+ represents the current state of the whole project. `git cat-file -t 11dd91`
244
+ tells us that it's a "tree," and `git cat-file -p 11dd91` shows
245
+ that it looks like this:
246
+
247
+ 100644 blob 81d4aaeff611587c378d25a90e1d2f178b484727 home.md
248
+ 040000 tree f9c298b9e62cecfc65ede840f48fa025347909fe images
249
+ 100644 blob ed91a30546d771a1c22ab2d1e846bdea6b52a624 software.md
250
+
251
+3. `.git/objects/31/8ff038bef2f31f61729fdebaaa02e8415c659a` (your name will be different; make a note of the hash you have) is a **commit** representing the new root for your repo. `git cat-file -t` tells us that it's a "commit."
252
+
253
+ The contents for mine are
254
+
255
+ tree 11dd91adca91dc2445dfcee4d3418e8ad219db7c
256
+ author John Norman <john@iorahealth.com> 1475414291 -0500
257
+ committer John Norman <john@iorahealth.com> 1475414291 -0500
258
+
259
+ Add pages and images
260
+
261
+Now we can trace this backwards. We want to do `git cat-file -t` and `git cat-file -p` on each of the following SHAs:
262
+
263
+* `318ff038bef2f31f61729fdebaaa02e8415c659a` **commit**
264
+* `11dd91adca91dc2445dfcee4d3418e8ad219db7c` **tree**: references blob home.md, tree images [`f9c298b9e62cecfc65ede840f48fa025347909fe`], and blob software.md)
265
+* `f9c298b9e62cecfc65ede840f48fa025347909fe` **tree**: references blob wat.jpg
266
+* `d2dd63f0a1e8f1c07f4c33e5af782a60fda3546f` **blob**: for wat.jpg
267
+
268
+Meanwhile, the contents of `.git/index` do not change. Tthis is because `.git/index` still represents what is
269
+being tracked, which, after a commit, is whatever is in the last commit -- i.e., `git diff --cached` returns nothing. `git status` will compare the state of the working directory with the index, and will find that there is one file that that is "untracked": `images.md`.
270
+
271
+### Making our entire repo with commands manipulating git objects directly
272
+
273
+Git provides for manipulating the object database directly, without using the commands to add to the index
274
+from your working directory. Here's the complete list of commands we used to set up our repo, add files,
275
+and do the two commits:
276
+
277
+ # Set up our directory and initialize the repo
278
+ cd $HOME
279
+ cd src
280
+ rm -fR wiki
281
+ mkdir wiki
282
+ cd wiki
283
+ echo "### Welcome to the wiki" >home.md
284
+ git init
285
+
286
+ # Add a file and commit
287
+ git add home.md
288
+ git commit -m "Initial commit"
289
+
290
+ # Create another file and add it
291
+ echo "Software we like: git, atom" > software.md
292
+ git add software.md
293
+
294
+ # Create an image directory, put a file in there, add the directory and image, and commit
295
+ mkdir images
296
+ curl -s http://i2.kym-cdn.com/photos/images/original/000/173/575/25810.jpg -o images/wat.jpg
297
+ git add images
298
+ git commit -m "Add pages and images"
299
+
300
+Now let's do all of this *without* creating any files in the working directory. We are going to
301
+add stuff to the index and `objects/` hierarchy directly.
302
+
303
+First, create our directory and initialize our repo.
304
+
305
+ cd $HOME
306
+ cd src
307
+ rm -fR wiki
308
+ mkdir wiki
309
+ cd wiki
310
+ git init
311
+
312
+We will now use the `hash-object` command with `-w` to create our object. Notice that no file is
313
+created in the working directory.
314
+
315
+ echo "### Welcome to the wiki" | git hash-object -w --stdin
316
+ # returns hash: 81d4aaeff611587c378d25a90e1d2f178b484727
317
+
318
+This only created the compressed object in `object/` -- we don't yet have anything in the index. To
319
+put a reference to our object in the index, we use `update-index`:
320
+
321
+ git update-index --add --cacheinfo 100644 81d4aaeff611587c378d25a90e1d2f178b484727 home.md
322
+
323
+The next thing we want to do is commit this object so that it is referenced from `HEAD`. Recall that
324
+the `commit` object needs to reference a `tree` object. Let's make a `tree` object from the current
325
+contents of the index. This command will report the hash for the newly-created tree.
326
+
327
+ git write-tree
328
+ # returns hash: 8dfcc6e23dec849d71d7428290cf66aab644a22f
329
+
330
+The hash value should look familiar from above.
331
+
332
+Now we will make the commit object with `commit-tree`:
333
+
334
+ echo "Initial commit" | git commit-tree 8dfcc6
335
+ # returns hash (yours will be different): 256b5975726098c52222d01787688681b959a991
336
+ # make a note of this hash: You will use it later
337
+
338
+At this point we can do `git log` if we pass in the commit hash
339
+
340
+ git log 256b5975726098c52222d01787688681b959a991
341
+
342
+Add `software.md` as above, but without creating a file in the working directory:
343
+
344
+ echo "Software we like: git, atom" | git hash-object -w --stdin
345
+ # hash = ed91a30546d771a1c22ab2d1e846bdea6b52a624
346
+ git update-index --add --cacheinfo 100644 ed91a30546d771a1c22ab2d1e846bdea6b52a624 software.md
347
+
348
+Add the binary for our image:
349
+
350
+ curl -s http://i2.kym-cdn.com/photos/images/original/000/173/575/25810.jpg | git hash-object -w --stdin
351
+ # returns hash: d2dd63f0a1e8f1c07f4c33e5af782a60fda3546f
352
+
353
+Add our file to the index and then make a new tree:
354
+
355
+ git update-index --add --cacheinfo 100644 d2dd63f0a1e8f1c07f4c33e5af782a60fda3546f images/wat.jpg
356
+ git write-tree
357
+ # returns hash: 11dd91adca91dc2445dfcee4d3418e8ad219db7c
358
+
359
+And commit. Note the additional `-p`: This specifies the prior commit so that this commit references
360
+the first one.
361
+
362
+ echo "Add pages and images" | git commit-tree 11dd91 -p 256b59
363
+ # returns hash (yours will be different): 914b2fef46cd6cd1c58c90def3a751d2fb738e39
364
+
365
+And now do git log with this new hash . . .
366
+
367
+ git log 914b2f
368
+
369
+Now let's create the master branch, and look at the log for that branch:
370
+
371
+ git update-ref refs/heads/master 914b2fef46cd6cd1c58c90def3a751d2fb738e39
372
+ # NOTE: Basically the same as . . .
373
+ # echo "914b2fef46cd6cd1c58c90def3a751d2fb738e39" > .git/refs/heads/master
374
+ git log --pretty=oneline master
375
+
376
+### What's in refs/?
377
+
378
+So now that we've committed something, we have a reference to the tip of our master branch. The SHA for the tip of our
379
+master branch is stored in the ordinary file `.git/refs/heads/master` -- the content is `318ff038bef2f31f61729fdebaaa02e8415c659a `, which
380
+is just the last thing we did with our repo: The last commit. Can we add another head? Yep. Do this:
381
+
382
+ git checkout -b jn-foobar
383
+
384
+Now we get a new file `.git/refs/head/jn-foobar`. And its content is also `318ff038bef2f31f61729fdebaaa02e8415c659a`, the SHA
385
+for our last commit. In other words, `master` and `jn-foobar` point to the same thing; meanwhile, `.git/HEAD` now is `jn-foobar`.
386
+
387
+We can play around with this endlessly, checking out `master`, then `jn-foobar` and see the content of `.git/HEAD` change accordingly.
388
+
389
+### What's in logs/?
390
+
391
+As you work, git will track your commits and keep them in various places in `logs/`. It is possible to reset
392
+your repo and lose references to hashes. But typically the compressed data is still somewhere in `objects/` --
393
+you just need to be able to find the hash for the lost file. By consulting the `logs/` via the `git reflog` command, you can sometimes recover lost work. By default, the log goes back 90 does for commits in your current history,
394
+and 30 days for commits that are no longer relevant.
395
+
396
+In my experience, if you find that you have to deal with the reflog, get help from someone who's dealt with
397
+it before.
398
+
399
+### What does it mean to do git add?
400
+
401
+When do you `git add`, you add a compressed version of the file(s) to the `objects/` directory (if it isn't there already), and add (a) line(s) to the `index`.
402
+
403
+### What's the opposite of git add?
404
+
405
+`git rm --cached filename`. Note, this will not delete the compressed binary version from the `objects/` directory.
406
+
407
+### What is the index? What is the "staging area"?
408
+
409
+git keeps a list of all of the files for the "next" commit. This list is called the `index`. It's also called the "staging area." It's also called the "cache." To get a list of all of the files in the index, type `git ls-files`. To get the same list, along with the hashes for each file and the file mode, do `git ls-files --stage`.
410
+
411
+It's all the files currently being tracked since the last commit, including removed files. So, for instance, if you clone a large repo such as what's at `git@github.com:IoraHealth/IoraHealth.git` and do `git ls-files | wc -l`, as of October 2016 you'll get 3913, which is the same the number of files in the repo not including what's in `.git/` (compare `find . -type f -not -path "./.git/*" | wc -l`).
412
+
413
+### What does git check -b branch do?
414
+
415
+### What does git commit do?
416
+
417
+### What does git reset do?
418
+
419
+### And what is the difference between git reset --hard and regular git reset?
420
+
421
+### What is the difference between git rm --cached [file] and git reset HEAD [file] and git reset [file]?
422
+
423
+### watch-do
424
+
425
+On the Mac,
426
+
427
+ brew install tree fswatch
428
+
429
+Create a file like this, call it `watch-do` and put it in your patch
430
+
431
+```
432
+#!/bin/bash
433
+
434
+# usage
435
+# watch-do <dir> <path-to-watch> <command-to-run>
436
+
437
+DIR=$1
438
+WATCH=$2
439
+COMMAND="clear && cd $DIR && $3"
440
+bash -c "$COMMAND"
441
+fswatch -l 0.1 -o "$WATCH" | xargs -n1 bash -c "$COMMAND"
442
+```
... ...
\ No newline at end of file