« Name poll! For something useless! Yay!
git problems for non-gits.
Is the admin area in urgent need of an overhaul? »

Nao

  • Dadman with a boy
  • Posts: 16,082
git problems for non-gits.
« on November 12th, 2013, 08:02 PM »Last edited on November 12th, 2013, 11:28 PM
Okay, maybe in the public area there are people who might know much more about git than I can... ;)
So this is a 'generic' topic for git-related issues. Feel free to use it for you, too.

I did my folder moves (for the theme removal stuff)[1], through a series of git mv old_long_folder/* new_short_folder/ commands, and it's working.

Now, when I right click Load.php (or any other file, really) and click Blame, TortoiseGit shows me a history of the file. I can click anywhere and go back to any old commits, so it's definitely doing git blame --follow. However, when I right-click a line and ask for 'Blame previous revision', it gives me this error:

Blame error
fatal: no such path sources/Load.php in (old commit ID)

Yes, of course that file wasn't there. But, Blame tool, you KNEW that, since you're listing all revisions of the file across renames! Git being able to find renames and code moves with needing to specifically track them is supposedly one of its biggest assets!
So, why do I have to go through that..?!
Am I condemned to go through filter-branch and tree-filter, and rename all files manually in the entire history...?! That would be an awful thing to do.

PS: I tested with Git Extensions and SmartGit, the first doesn't even react when I ask for an earlier commit, while the second one will show me earlier commits, so that's great, but the tool itself, I'm not really convinced by it, and I'm definitely sticking with TortoiseGit.
 1. BTW, I'm unsure whether I'll keep both 'scripts' and 'sources' folders side to side in the root. I tend to click on 'scripts' when wanting to view PHP files, and 'sources' when wanting to see JS files, so... Maybe it's just me, but if any of you has a suggestion on renaming the 'scripts' folder to something short and sweet and understandable, knowing that the name 'js' is already taken by the JS cache, I'm very open about this.
Re: git problems for non-gits.
« Reply #2, on November 13th, 2013, 04:47 PM »
Issue apparently got acknowledged by a contributor, so it does mean it's not a wrong operation on my part; it's something that others managed to reproduce. Good!

Also opened another issue, more like a suggestion:
https://code.google.com/p/tortoisegit/issues/detail?id=1990

I don't like making too many suggestions for software I don't (or in this case, can't) contribute to, but these are the two items that directly affect the viability of the Wedge git repo. Former causes Blame to fail everywhere after my latest commits, and latter causes two extra 'steps' when blaming an old file, because in SVN, I once committed many files with the Windows whitespace (used the wrong search-replace software and didn't check!), and then I redid a commit to strip the extra \r bytes.
(I guess I could use a rebase to reduce this by at least one step, but... I'm not sure if it's even doable.)
Re: git problems for non-gits.
« Reply #3, on November 16th, 2013, 11:00 PM »
Regarding issue 1990, I decided to look into squashing the commit with its fix.
So, I made a backup of my repo, then tried various options in TortoiseGit, which all failed. Nice. I then used the bash, with git rebase -i, and it was incredibly, horribly not natural. For instance, I can't select the buggy commit and rebase from that. I have to select the commit BEFORE it, and then ignore the buggy commit, and change the bugfix to 'squash'.
Then I can save the rebase file, wait for 10 minutes (how slow!), and I get a rewritten history.
Only problem, but I was expecting that, is that all subsequent commits have a different SHA1, so I'll/I'd have to push --force the result, and expect all of you guys to give up on your current repo, clone the updated repo somewhere else, and then transmit all your changes/commits to that new repo. A bit annoying...
But, OTOH, this also eliminates the 2 extra 'blame previous file' steps that we currently get everytime we're trying to blame a file that has an instance of a we::$user call (all of these ~99 files had their line endings corrupted by this wide-scale change using new software. Thank you, new software. Well, it's old and forgotten software since that day, of course.)

Are you okay with that, guys..?
I can't really expect the TortoiseGit committers to suddenly offer a -w parameter on their blame tool's settings UI, and if they do, I don't know if it'll be saved for all uses, etc, so I'd rather fix the repo myself.
I could also use the opportunity to rebase other problems in the history, but... Actually, I can't really think of anything else, so it's just this one shot, normally.
Re: git problems for non-gits.
« Reply #4, on November 16th, 2013, 11:26 PM »
Oh, yes I know, I could filter-branch to move all files to their 'final destinations', but I don't think I'll enjoy the process at all, so... Let's forget about that really, I have more hopes that the TortoiseGit will fix that one's issue in a timely manner ;)
Posted: November 16th, 2013, 11:06 PM

Another thing. I hate that when I stash files, when they pop back into existence, their file dates are all reset to right now. You can't possibly expect me to use a system that screws up my entire conception of a repo folder, i.e. the most commonly modified files are the first ones to show up (sorted by modified date.)
Meaning, I have to copy my stashable files to another folder, and then do the whole process of copying them back later, etc... Very, very annoying. I don't know if it's a TortoiseGit issue, but I doubt it's due to something else than git itself.
Re: git problems for non-gits.
« Reply #5, on November 19th, 2013, 12:25 PM »
I think the merge workflow won't work for Wedge. See latest commit. I do appreciate having graphs with branches etc, but (1) I'm not sure I like the extra commit for branch merges, and (2) I have this feeling that merging requires doing serious branching work, and I'm... really, really not that much into branching for everyday work. Or even for any kind of work. I just systematically work on multiple features at the time in one file, and am used to simply committing partial versions of these files, then restoring them, etc. It's my workflow. It looks like it's closer to what a rebase workflow would work with.
Am I wrong to assume that?
And, how would I 'efficiently' go for a rebase workflow (now that I *know* that rebasing is pretty much okay), while still keeping the pull request system? Is it even possible?

PS: the TortoiseGitBlame bug was fixed. :) Can't wait for the next preview release, yay!
Re: git problems for non-gits.
« Reply #6, on December 15th, 2013, 10:58 PM »
I tried using the pull request workflow from Github on my fix-globals tool.

What a complicated task, if you'll ask me... :-/

- Clicked Fork on the Wedge/tools repo
- Updated my local repo to add an origin remote to my Nao/tools repo, rename the Wedge/tools repo to upstream.
- Made my local commit. (Forgot to make it to a new branch... Oh well, whatever. One day, I'll remember.)
- Pushed to origin. By default, it wanted me to push to upstream.. Well, no.
- Went back to the website. Clicked Pull Request. Filled in the details, then realized I'd made a typo...
- Went back to local, 'amend last commit', fixed changelog, committed, pushed (again, to origin).
- Went back to website, pull request, submitted.
- Went to the upstream repo pull request page, commented (I have to say, the design is really confusing, compared to Bitbucket.)
- Clicked Merge Pull Request.
- Went back to origin repo, it said the repo was up to date. Eh, really..? But it doesn't have the merge commit?!
- Went back to local, pulled from upstream, it added the merge commit. Then pushed to origin.
- Went back to origin repo, and now it has the proper history.

So, that means:
- I didn't have to play with rebase at any point. Does that mean I'm free from it, or not? Is rebasing needed only when committing from another branch...?
- There's an AWFUL LOT of going back and forth between my local repo and my two remote repos.

I was thinking of using that system for Wedge commits in the future (once it's on github)[1], but I don't know if I'm willing to bother, really. Basically, if I consider my local repo to be the 'official' one, then I guess the workflow is completely different, and I'll have to learn it (again).

What do you think, guys..? Anyone experienced enough to tell whether it's 'proper', or there's an easier way to do things, or it just gets easy once you do it enough times?

Why the hell does the 'Signed-off' mention only show up in the extended log (i.e. I have to click the '...'), meaning that if I want to check for sign-off status, I have to click Commits, then for each commit open the ..., and pray that it's there..? Why can't github 'simply' add a 'signed-off' *icon* next to the commit entry, so I can immediately tell if it's okay..?
 1. Maybe with the hope that I can use Nao/Wedge as my proper work place, and Wedge/Wedge as the place for stable code to be moved to, so that I can afford messing up my pushes and force-pushing them again, with the hope that no one will be forking my own work repo..????

emanuele

  • Posts: 125
Re: git problems for non-gits.
« Reply #7, on December 17th, 2013, 11:05 PM »
Quote from Nao on December 15th, 2013, 10:58 PM
- I didn't have to play with rebase at any point. Does that mean I'm free from it, or not? Is rebasing needed only when committing from another branch...?
If you are doing things "right" you should *never* use rebase (unless of merge conflicts, but that's another topic).
You may have to use rebase if you commit to your local repo at master, and the pull it because git may bet confused by the new commit that is the merge of the pull request.
Or it may not, because when you pull you just pull the merge commit (because in fact the other commit is just the same as the one you did).
Quote from Nao on December 15th, 2013, 10:58 PM
Basically, if I consider my local repo to be the 'official' one, then I guess the workflow is completely different, and I'll have to learn it (again).
You can work with a sort of "centralized" repo, yes (that is basically what the "upstream" in your case is), the only problem I can see is this:
* repo upstream => commit 1 (let's say)
* I do a commit in my local and push => repo emanuele commit 2
* in the meantime you do a commit on your local repo => origin commit 3
* I send a PR and you merge it from github => repo upstream commit => 4
Now, you'd have to synch your local repo and the remote one (upstream) but: if you push you get a fast-forward (or whatever it is) error because you have new commits locally and new commits in the remote, so you have first to "pull"[1] the remote repo and then push it again.

It's not so different in fact (and probably for your work-flow it could make more sense), the only difference is what is considered the "master" repository: doing the pull request way is the one that belongs to he organization (like for Elk and SMF, in this case you never commit to the org repo, but always to your local one and then synch), in the other you consider your local repo the original one and you could even ignore PR at github and instead fetch the other repos and do the merges directly on your local copy (sorry, I'm one of the few that apparently loves git workflow and can't stop thinking about all the alternatives to do the same thing LOL).
Quote from Nao on December 15th, 2013, 10:58 PM
Why the hell does the 'Signed-off' mention only show up in the extended log (i.e. I have to click the '...'), meaning that if I want to check for sign-off status, I have to click Commits, then for each commit open the ..., and pray that it's there..? Why can't github 'simply' add a 'signed-off' *icon* next to the commit entry, so I can immediately tell if it's okay..?
Good question... you should suggest it to them.
Probably because sign-off is not *so* popular...I think, but dunno.
 1. pull is equivalent to fetch + merge, so "grab the remote history and merge it into your local branch"

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: git problems for non-gits.
« Reply #8, on December 29th, 2013, 12:02 PM »
Quote from emanuele on December 17th, 2013, 11:05 PM
Quote from Nao on December 15th, 2013, 10:58 PM
- I didn't have to play with rebase at any point. Does that mean I'm free from it, or not? Is rebasing needed only when committing from another branch...?
If you are doing things "right" you should *never* use rebase (unless of merge conflicts, but that's another topic).
AFAIK, rebase is pretty much the same as merge, it's just the way it shows up in the history log that changes..? I mean, in terms of merging the actual files together, etc.

I've got so many problems with merging, eh... For instance, if merging on bitbucket or github, you get an extra commit with the merge process. The infamous ones, the ones that take up all of the space in the history log, and make it look like there's more activity than usual. If merging locally, nothing happens. It's just a fast-forward, so it simply moves the ref for that branch to the other branch's ref. AFAICT, at least.

Now, here's the process I used recently for my workflow...
- master
- Committed theme removal code. Decided to use 'new branch' in the commit window, typed 'byebye-themes'.
- Worked on that branch, then I wanted to switch to PHP 5.3-only. Instead of stashing, switching to master, unstashing, then committing to a new branch (WTF?!), I simply did another 'new branch' in the commit window, and typed 'php-5.3', and committed my changes.
- Wanted to come back to byebye-themes to commit my theme-related changes, but it didn't want to switch, because of the uncommitted stuff. First problem: why the hell can I commit to a 'new branch', but NOT to an existing, prior branch...?! Why would I need to stash, then checkout, then unstash, then uncommit?! I think I could use cherry-picking to move an existing commit to a prior branch, but I'm not very familiar with that, and I don't know if it's the 'recommended way'.
- Okay, now, it gets more complicated... If I try to delete the byebye-themes branch, it tells me that HEAD doesn't have all of the commits in it (!), even though I'd created the new branch from that one. What I want to do at this point, is merge everything back into master.
- I thought, okay, I'll just merge branches manually, one by one. So I went to the master branch, then requested a merge from byebye-themes. (I also tried rebase, with the same effect.) It told me there were conflicts. Which is HORRIBLY WRONG, because byebye-themes was committed FROM master, and master was 100% clean after that commit, so a simple fast-forward should have been enough, but it still found 40+ completely silly merge conflicts, which I'm not willing to fix manually.

I mean, if it's all that complicated, I'm NEVER going to use branches again. And you know what that means..? It means I'm just doing SVN stuff, and then I'll start requesting that people send me patches, instead of pull requests. I know that many popular projects 'do it' this way, so it's not a big deal in the end, but wasn't the whole branching business the main selling point of git..? Wasn't its 3-way merge lauded as a genius thing? Why then, would I get a merge conflict on something that just needs a fast-forward?
Quote from emanuele on December 17th, 2013, 11:05 PM
You can work with a sort of "centralized" repo, yes (that is basically what the "upstream" in your case is), the only problem I can see is this:
* repo upstream => commit 1 (let's say)
* I do a commit in my local and push => repo emanuele commit 2
* in the meantime you do a commit on your local repo => origin commit 3
* I send a PR and you merge it from github => repo upstream commit => 4
Now, you'd have to synch your local repo and the remote one (upstream) but: if you push you get a fast-forward (or whatever it is) error because you have new commits locally and new commits in the remote, so you have first to "pull"[1] the remote repo and then push it again.
 1. pull is equivalent to fetch + merge, so "grab the remote history and merge it into your local branch"
(Why is everybody so frigging happy to use git when it's just systematically the same mess, and with SVN around the corner, they could just choose to forego all of that crap?)
Quote from emanuele on December 17th, 2013, 11:05 PM
It's not so different in fact (and probably for your work-flow it could make more sense), the only difference is what is considered the "master" repository: doing the pull request way is the one that belongs to he organization (like for Elk and SMF, in this case you never commit to the org repo, but always to your local one and then synch), in the other you consider your local repo the original one and you could even ignore PR at github and instead fetch the other repos and do the merges directly on your local copy (sorry, I'm one of the few that apparently loves git workflow and can't stop thinking about all the alternatives to do the same thing LOL).
I actually tried to do that with Pandos's repo, so that I wouldn't have to bother with duplicate merges. However, now every time I try to commit, it asks me WHERE I should commit. It doesn't accept a 'default' remote, it wants to know which remote every time, so I'm going to have to remove all of my extra remotes. Pff...
Quote from emanuele on December 17th, 2013, 11:05 PM
Good question... you should suggest it to them.
Probably because sign-off is not *so* popular...I think, but dunno.
I doubt everyone's using a DCO..? :-/
Or maybe pull requests are considered, by practice, to be a de facto sign-off..?
Re: git problems for non-gits.
« Reply #9, on December 29th, 2013, 01:19 PM »
I think I've found the reason for the merge conflicts in master...

Apparently, here's how it happened:

- 14 December: committed my theme changes to 'master'.
- 15 December: fixed many bugs. Decided that theme removal was worth a branch by itself. Clicked 'Amend' on the commit window, so that it would re-use the last commit. Click 'new branch', entered 'byebye-themes'.

- As a result, the 'master' branch was NOT reset to its previous state. Instead, it still holds the FIRST version of the commit, dated from December 14, while byebye-themes has the second version of it, also dated December 14, but if you click 'Commit Date', you'll see it's actually from December 15.

- So, attempting to merge byebye-themes into master WOULD result in conflicts. What I'd need to do first is reset --hard the master branch to origin/master, or basically, merge my local byebye-themes into origin/master. Or, remove the master branch, and then rename the php-5.3 branch to master. What is best..?

emanuele

  • Posts: 125
Re: git problems for non-gits.
« Reply #10, on December 29th, 2013, 02:16 PM »
Quote from Nao on December 29th, 2013, 12:02 PM
AFAIK, rebase is pretty much the same as merge, it's just the way it shows up in the history log that changes..? I mean, in terms of merging the actual files together, etc.
Rebase is a completely different thing from merge. Really, two different beasts.
Merge (or pull) does a single thing: brings the code from a branch into another branch. Rebase allows you to rewrite the repo history.
Of course rewrite history is "dangerous" (especially if you do it on master because it changes everything for everybody), but in certain cases it is necessary. I have the terrible habit to do git commit -asm "message", well that works when you know what you have done, but when you already have some previous non-committed stuff floating around it may create not-so-nice commits, with rebase (interactive rebase actually) you can go back to a certain commit, rework the commit, split it, squash two into one, remove unnecessary ones, etc. I do that things "frequently", for example every time I start playing with Travis-ci and new tests and I end up testing a lot of random commits just to see what is wrong, but bring that stuff into master would be stupid, so when I'm done, I come back and rework the branch to have a good-looking series of commits with a rebase.
Quote from Nao on December 29th, 2013, 12:02 PM
Now, here's the process I used recently for my workflow...
- master
- Committed theme removal code. Decided to use 'new branch' in the commit window, typed 'byebye-themes'.
- Worked on that branch, then I wanted to switch to PHP 5.3-only. Instead of stashing, switching to master, unstashing, then committing to a new branch (WTF?!), I simply did another 'new branch' in the commit window, and typed 'php-5.3', and committed my changes.
- Wanted to come back to byebye-themes to commit my theme-related changes, but it didn't want to switch, because of the uncommitted stuff. First problem: why the hell can I commit to a 'new branch', but NOT to an existing, prior branch...?! Why would I need to stash, then checkout, then unstash, then uncommit?! I think I could use cherry-picking to move an existing commit to a prior branch, but I'm not very familiar with that, and I don't know if it's the 'recommended way'.
- Okay, now, it gets more complicated... If I try to delete the byebye-themes branch, it tells me that HEAD doesn't have all of the commits in it (!), even though I'd created the new branch from that one. What I want to do at this point, is merge everything back into master.
- I thought, okay, I'll just merge branches manually, one by one. So I went to the master branch, then requested a merge from byebye-themes. (I also tried rebase, with the same effect.) It told me there were conflicts. Which is HORRIBLY WRONG, because byebye-themes was committed FROM master, and master was 100% clean after that commit, so a simple fast-forward should have been enough, but it still found 40+ completely silly merge conflicts, which I'm not willing to fix manually.
I feel the problem is simply that git is not SVN.
You are trying to use it thinking (maybe in your subconscious) they are the same thing and they work the same, but it's not the case. ;)

In SVN branches are different directories, in git no. They are different versions of the same directory.
When you think about it it's obvious that to switch from a branch to another you cannot have uncommitted stuff. Uncommitted stuff are things that git doesn't yet know where they should be, so you have to tell it that you want them into a certain branch, otherwise it will switch to any branch you are switching to (that is not a bad thing per se, just a bit unreliable, but I use it from time to time).
Quote from Nao on December 29th, 2013, 12:02 PM
(Why is everybody so frigging happy to use git when it's just systematically the same mess, and with SVN around the corner, they could just choose to forego all of that crap?)
I probably adapted my workflow to git rather easily because I never really used SVN, just about 20 commits to SMF's SVN repo, and I wasn't really happy with SVN logic (or non-logic IMHO).
Quote from Nao on December 29th, 2013, 12:02 PM
I actually tried to do that with Pandos's repo, so that I wouldn't have to bother with duplicate merges. However, now every time I try to commit, it asks me WHERE I should commit. It doesn't accept a 'default' remote, it wants to know which remote every time, so I'm going to have to remove all of my extra remotes. Pff...
mmm... why do you want to "commit" to a remote?
You push to a remote, but commit locally.
Quote from Nao on December 29th, 2013, 12:02 PM
I doubt everyone's using a DCO..? :-/
Or maybe pull requests are considered, by practice, to be a de facto sign-off..?
Yep, DCO are not so used simply because commits are usually considered enough of an acceptance of the license and other related stuff (TBH I've been thinking to propose to drop he DCO for Elk) and in fact they don't even have such a big "legal" relevance IMHO (always considering I'm not a lawyer) because anyway most of us are signing with just a nick (and I suspect I'm one of the few using a valid email either :P).
The DCO was introduced in SMF just as a way to calm SM's fear of losing some kind of "property" of the code when it was clear that ask for a CLA was a no-go for github contributions, but in fact it doesn't really give anything to anyone[1], it's just a "confirmation" that you know what you are doing and you can do it, nothing else.
Quote from Nao on December 29th, 2013, 01:19 PM
- So, attempting to merge byebye-themes into master WOULD result in conflicts.
I suspected it was something like that...
Quote from Nao on December 29th, 2013, 01:19 PM
What I'd need to do first is reset --hard the master branch to origin/master, or basically, merge my local byebye-themes into origin/master. Or, remove the master branch, and then rename the php-5.3 branch to master. What is best..?
Both should work, I tend to prefer the first (that is what I usually do), more clean.
 1. and on that there was a fight too with people (from SM) accusing others (developers mainly) to have masked (purposefully) the real meaning of the DCO, etc, but that's completely OT.

MultiformeIngegno

  • Posts: 1,337
Re: git problems for non-gits.
« Reply #11, on December 29th, 2013, 04:07 PM »
Little OT: « Reply #10 108 minutes ago » doesn't make much sense... wouldn't it be better to have « Reply #10 1 hour and 48 minutes ago » ..?
« Name poll! For something useless! Yay!
Is the admin area in urgent need of an overhaul? »