Maintaining Git trees

Obtaining the current media work tree

Up to date instructions can be found at http://git.linuxtv.org/media_tree.git. They currently read as follows (replace *X.X* with a future kernel version number, typically two versions ahead of the current kernel):

To clone the master development repository, install git, and run:

 git clone git://github.com/torvalds/linux.git v4l-dvb
 cd v4l-dvb
 git remote add linuxtv git://linuxtv.org/media_tree.git
 git remote update
 git checkout -b media-master remotes/linuxtv/staging/for_v*X.X*

If you want to clone a different repository, just replace the URL at the git remote add line above.

In order to update your repository, you should do:

 git pull . remotes/linuxtv/staging/for_v*X.X*

And to put your work on the top of a stanging branch:

 git rebase remotes/linuxtv/staging/for_v*X.X*

KERNEL DEVELOPMENT PROCEDURES AT THE KERNEL UPSTREAM

It is important that people understand how upstream development works.

Kernel development has 2 phases:

1. a merge window, typically 2 weeks long, starting with the release of a new kernel version;
2. the -rc period, where the new kernel is tested and receives fixes.

The length of the -rc period depends on the number and relevance of the fixes. There are usually between 6 and 9 -rc releases with a week between each, but it can vary. In general, the announcement of a new -rc kernel gives some hints when that -rc kernel may be the last one.

Subsystem procedures for merging patches upstream

The required procedure on subsystem trees is that:

a) During -rc period (e.g. latest main kernel available is 3.x, the latest -rc kernel is 3.[x+1]-rc<y>):

• fix patches for the -rc kernel (3.[x+1]) should be sent upstream, being a good idea to send them for some time at linux-next tree, allowing other people to test it, and check for potential conflicts with the other arch's;
• patches for 3.[x+2] should be sent to linux-next.

b) the release of 3.[x+1] kernel:

• closes the -rc period and starts the merge window.

c) During the merge window:

• the patch that were added on linux-next during the -rc period for 3.[x+2] should be sent upstream;
• new non-fix patches should be hold until the next -rc period starts, so, they'll be added on 3.[x+3];
• fix patches for 3.[x+2] should go to linux-next, wait for a few days and then send upstream.

d) the release of 3.[x+2]-rc1 kernel:

• the merge window has closed. No new features are allowed.
• the patches with new features that arrived during the merge window will be moved to linux-next

A practical example

Considering that, at the time this document were written, the last main release is 2.6.32, and the latest -rc release is 2.6.33-rc5, this means that:

• Stable patches, after adding upstream, are being received for 2.6.32 kernel series;
• Bug fixes are being received for kernel 2.6.33;
• New feature patches are being received for kernel 2.6.34.

After the release of kernel 2.6.33, starts the period for receiving patches for 2.6.35.

In other words, the features being developed are always meant to be included on the next 2 kernels.

In the specific case of new drivers that don't touch on existing features, it could be possible to send it during the -rc period, but it is safer to assume that those drivers should follow the above procedure, as a later submission may be nacked.

Patches for stable

Sometimes, a fix patch corrects a problem that happens also on stable kernels (e. g. on kernel 3.x or even 3.y, where y < x). In this case, the patch should be sent to stable@kernel.org, in order to be added on stable kernels.

In the case of git-submitted patches with fixes, that also need to be send to stable, all the developer needs to do is to add to the patch description:

 CC: stable.kernel.org

At the moment the patch reaches upstream, a copy of the patch will be automatically be sent to the stable maintainer and will be considered for inclusion on the next stable kernel (3.x.y).

KERNEL DEVELOPMENT PROCEDURES FOR V4L/DVB

The upsteam procedures should be followed by every kernel subsystem. The subsystems have their own specific procedures detailing how the development patches are handled before arriving upstream. In the case of v4l/dvb, those are the used procedures.

Fixes and linux-next patches

One of the big problems of our model used in the past by the subsystem, based on one Mercurial tree, is that there were just one tree/branch for everything. This makes hard to send some fix patches for 2.6.[x+1], as they may have conflicts with the patches for 2.6.[x+2]. So, when the conflict is simple to solve, the patch is sent as fixes. Otherwise, the patch generally needed to hold to the next cycle. The fix patches used to get a special tag, added by the developer ("Priority: high", in the body of the description), to give a hint to the subsystem maintainer that the patch should be sent upstream.

Unfortunately, sometimes people mark the driver with the wrong tag. For example, a patch got merged on Jan, 22 2010 that marked with "high". However, that patch didn't apply at the fixes tree, because it fix a regression introduced by a driver that weren't merged upstream yet.

How to solve those issues?

Well, basically, the subsystem should work with more than one tree (or branch), on upstream submission:

• a tree(branch) with the fix patches;
• a tree(branch) with the new feature patches.

So, the subsystem uses two development -git trees:

• http://linuxtv.org/git//v4l-dvb.git - for patches that will be sent to the [x+2] kernel (and merged at upstream linux-next tree)
• http://linuxtv.org/git//fixes.git - for bug patches that will be sent to the [x+1] kernel (also, patches that need to go to both [x+1] and [x])

While the patches via -hg, due to the merge conflicts its mentioned, the better is that, even those developers that prefer to develop patches use the old way, to send the fix patches via -git. This way, if is there a conflict, he is the one that can better solve it. Also, it avoids the risk of a patch being wrongly tagged.

Also, after having a patch added on one of the above trees, it can't simply remove it, as others will be cloning that tree. So, the only option would be to send a revert patch, causing the patch history to be dirty and could be resulting on some troubles when submitting upstream. I've seen some nacks on receiving patches upstream from dirty git trees. So, we should really avoid this.

how to submit a -git pull request

As the same git tree may have more than one branch, and we'll have 2 -git trees for upstream, it is required that people specify what should be done. The internal maintainer's workflow is based on different mail queues for each type of requesting received.

There are some scripts to automate the process, so it is important that everyone that sends -git pull do it at the same way.

So, a pull request to be send with the following email tags:

 From: <your real email>
 Subject: [GIT FIXES FOR 2.6.33] Fixes for driver cx88
 To: linux-media@vger.kernel.org

 From: <your real email>
 Subject: [GIT PATCHES FOR 2.6.34] Updates for the driver saa7134
 To: linux-media@vger.kernel.org

The from line may later be used by the git mailbomb script to send you a copy when the patch were committed, so it should be your real email.

The indication between [] on the subject will be handled by the mailer scripts to put the request at the right queue. So, if tagged wrong, it may not be committed.

Don't send a copy of the pull to the maintainer addresses. The pull will be filtering based on the subject and on the mailing list. If you send a c/c to the maintainer, it will be simply discarded.

NEVER send a copy of any pull request to a subscribers-only mailing list. Everyone is free to answer to the email, reviewing your patches. Don't penalty people that wants to contribute with you with SPAM bouncing emails, produced by subscribers only lists.

When a patch touches on other subsystem codes, please copy the other subsystem maintainers. This is important for patches that touches on arch files, and also for -alsa non-trivial patches.

The email should be generated with the usage of git request-pull:

      git request-pull $ORIGIN $URL

where $ORIGIN is the commit hash of the tree before your patches, and $URL is the URL for your repository.

For example, for the patches merged directly from -hg at the -git trees on Jan, 22 2010, the above commands produced:

 The following changes since commit 2f52713ab3cb9af2eb0f9354dba1421d1497f3e7:
   Abylay Ospan (1):
         V4L/DVB: 22-kHz set_tone fix for NetUP Dual DVB-S2-CI card. 22kHz logic controlled by demod
 
 are available in the git repository at:
 
   git://linuxtv.org/v4l-dvb.git master
 
 Andy Walls (4):
       V4L/DVB: cx25840, v4l2-subdev, ivtv, pvrusb2: Fix ivtv/cx25840 tinny audio
       V4L/DVB: ivtv: Adjust msleep() delays used to prevent tinny audio and PCI bus hang
       V4L/DVB: cx18-alsa: Initial non-working cx18-alsa files
       V4L/DVB: cx18-alsa: Add non-working cx18-alsa-pcm.[ch] files to avoid data loss
 
 Devin Heitmueller (20):
       V4L/DVB: xc3028: fix regression in firmware loading time
       V4L/DVB: cx18: rename cx18-alsa.c
       V4L/DVB: cx18: make it so cx18-alsa-main.c compiles
       V4L/DVB: cx18: export a couple of symbols so they can be shared with cx18-alsa
       V4L/DVB: cx18: overhaul ALSA PCM device handling so it works
       V4L/DVB: cx18: add cx18-alsa module to Makefile
       V4L/DVB: cx18: export more symbols required by cx18-alsa
       V4L/DVB: cx18-alsa: remove unneeded debug line
       V4L/DVB: cx18: rework cx18-alsa module loading to support automatic loading
       V4L/DVB: cx18: cleanup cx18-alsa debug logging
       V4L/DVB: cx18-alsa: name alsa device after the actual card
       V4L/DVB: cx18-alsa: remove a couple of warnings
       V4L/DVB: cx18-alsa: fix memory leak in error condition
       V4L/DVB: cx18-alsa: fix codingstyle issue
       V4L/DVB: cx18-alsa: codingstyle fixes
       V4L/DVB: cx18: codingstyle fixes
       V4L/DVB: cx18-alsa: codingstyle cleanup
       V4L/DVB: cx18-alsa: codingstyle cleanup
       V4L/DVB: cx18: address possible passing of NULL to snd_card_free
       V4L/DVB: cx18-alsa: Fix the rates definition and move some buffer freeing code.
 
 Ian Armstrong (1):
       V4L/DVB: ivtv: Fix race condition for queued udma transfers
 
 Igor M. Liplianin (4):
       V4L/DVB: Add Support for DVBWorld DVB-S2 PCI 2004D card
       V4L/DVB: dm1105: connect splitted else-if statements
       V4L/DVB: dm1105: use dm1105_dev & dev instead of dm1105dvb
       V4L/DVB: dm1105: use macro for read/write registers
 
 JD Louw (1):
       V4L/DVB: Compro S350 GPIO change
 
  drivers/media/common/tuners/tuner-xc2028.c  |   11 +-
  drivers/media/dvb/dm1105/Kconfig            |    1 +
  drivers/media/dvb/dm1105/dm1105.c           |  501 ++++++++++++++-------------
  drivers/media/video/cx18/Kconfig            |   11 +
  drivers/media/video/cx18/Makefile           |    2 +
  drivers/media/video/cx18/cx18-alsa-main.c   |  293 ++++++++++++++++
  drivers/media/video/cx18/cx18-alsa-mixer.c  |  191 ++++++++++
  drivers/media/video/cx18/cx18-alsa-mixer.h  |   23 ++
  drivers/media/video/cx18/cx18-alsa-pcm.c    |  353 +++++++++++++++++++
  drivers/media/video/cx18/cx18-alsa-pcm.h    |   27 ++
  drivers/media/video/cx18/cx18-alsa.h        |   59 ++++
  drivers/media/video/cx18/cx18-driver.c      |   40 ++-
  drivers/media/video/cx18/cx18-driver.h      |   10 +
  drivers/media/video/cx18/cx18-fileops.c     |    6 +-
  drivers/media/video/cx18/cx18-fileops.h     |    3 +
  drivers/media/video/cx18/cx18-mailbox.c     |   46 +++-
  drivers/media/video/cx18/cx18-streams.c     |    2 +
  drivers/media/video/cx25840/cx25840-core.c  |   48 ++-
  drivers/media/video/ivtv/ivtv-irq.c         |    5 +-
  drivers/media/video/ivtv/ivtv-streams.c     |    6 +-
  drivers/media/video/ivtv/ivtv-udma.c        |    1 +
  drivers/media/video/pvrusb2/pvrusb2-hdw.c   |    1 +
  drivers/media/video/saa7134/saa7134-cards.c |    4 +-
  include/media/v4l2-subdev.h                 |    1 +
  24 files changed, 1380 insertions(+), 265 deletions(-)
  create mode 100644 drivers/media/video/cx18/cx18-alsa-main.c
  create mode 100644 drivers/media/video/cx18/cx18-alsa-mixer.c
  create mode 100644 drivers/media/video/cx18/cx18-alsa-mixer.h
  create mode 100644 drivers/media/video/cx18/cx18-alsa-pcm.c
  create mode 100644 drivers/media/video/cx18/cx18-alsa-pcm.h
  create mode 100644 drivers/media/video/cx18/cx18-alsa.h

This helps to identify what's expected to be found at the -git tree and to double check if the merge happened fine.

Tags that a patch receive after its submission

This is probably the most complex issue to solve.

Signed-off-by/Acked-by/Tested-by/Nacked-by tags may be received after a patch or a -git submission. This can happen even while the patch is being tested at linux-next, from people reporting problems on the existing patches, or reporting that a patch worked fine.

Also, the driver maintainer and the subsystem maintainer that is committing those patches should sign each one, to indicate that he reviewed and has accepted the patch.

Currently, if a new tag is added to a committed patch, its hash will change. There were some discussions at Linux Kernel Mailing List about allowing adding new tags on -git without changing the hash, but I think this weren't implemented (yet?).

The same problem occurs with -hg, but, as -hg doesn't support multiple branches (well, it has a "branch" command, but the concept of branch there is different), it was opted that the -hg trees won't have all the needed SOBs. Instead, those would be added only at the submission tree.

With -git, a better procedure can be used:

The developer may have two separate branches on his tree. For example, let's assume that the developer has the following branches on his tree:

• media-master (associated with "linuxtv" remote)
• fixes
• devel

His development happens on devel branch. When the patches are ready to submission will be copied into a new for_submission branch: git branch for_submission devel

And a pull request from the branch "for_submission" will be sent.

Eventually, he'll write new patches on his devel branch.

After merged, the developer updates the linuxtv remote and drops the for_submission branch. This way, "media-master" will contain his patches that got a new hash, due to the maintainer's SOB. However, he has some new patches on his devel, that applies over the old hashes.

Fortunately, git has a special command to automatically remove the old objects: git rebase.

All the developer needs to do is to run the commands bellow:

git command	notes
git remote update	to update his remotes, including "linuxtv"
git checkout devel	move to devel branch
git pull . media-master	to make a recursive merge from v4l/dvb upstream
git rebase media-master	to remove the legacy hashes

After this, his development branch will contain only upstream patches + the new ones he added after sending the patches for upstream submission.

Patches submitted via email

All valid patches submitted via email to Linux Media ML are automatically stored at Patchwork. A patch, to be valid, should be in diff unified format. If you're using a -git tree, the simplest way to generate unified diff patches is to run:

 git diff

If you're writing several patches, the better is to create a tag or a branch for the changes you're working. After that, you can use

  git format-patch <origin_branch>

to create the patches for email submission.

Example

Suppose that the -git tree were created with:

 git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git v4l-dvb
 cd v4l-dvb
 git remote add linuxtv git://linuxtv.org/media_tree.git
 git remote update
 git checkout -b media-master remotes/linuxtv/staging/for_v3.6

Before start working, you need to create your work branch:

 git branch work media-master

And move the working copy to the "work" branch:

 git checkout work

Some changes were done at the driver and saved by commit:

 git commit -as

When the patches are ready for submission via email, all that is needed is to run:

 git format-patch work

The command will create a series of emails bodies, one file per email.

Just send the email with the patch inlined for it to get caught by patchwork.

BE CAREFUL: several emailers including Thunderdird breaks long lines, causing patch corruption.