Both qc_prep functions deal only with BMDMA PRD setup and PIO only SFF
drivers don't need them.  Rename to ata_bmdma_[dumb_]qc_prep() and
relocate.

All usages are renamed except for pdc_adma and sata_qstor.  Those two
drivers are not BMDMA drivers and don't need to call BMDMA qc_prep
functions.  Calls to ata_sff_qc_prep() in the two drivers are removed.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

Some of error handling logic in ata_sff_error_handler() and all of
ata_sff_post_internal_cmd() are for BMDMA.  Create
ata_bmdma_error_handler() and ata_bmdma_post_internal_cmd() and move
BMDMA part into those.

While at it, change DMA protocol check to ata_is_dma(), fix
post_internal_cmd to call ap->ops->bmdma_stop instead of directly
calling ata_bmdma_stop() and open code hardreset selection so that
ata_std_error_handler() doesn't have to know about sff hardreset.

As these two functions are BMDMA specific, there's no reason to check
for bmdma_addr before calling bmdma methods if the protocol of the
failed command is DMA.  sata_mv and pata_mpc52xx now don't need to set
.post_internal_cmd to ATA_OP_NULL and pata_icside and sata_qstor don't
need to set it to their bmdma_stop routines.

ata_sff_post_internal_cmd() becomes noop and is removed.

This fixes p3 described in clean-up-BMDMA-initialization patch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

port_task is tightly bound to the standard SFF PIO HSM implementation.
Using it for any other purpose would be error-prone and there's no
such user and if some drivers need such feature, it would be much
better off using its own.  Move it inside CONFIG_ATA_SFF and rename it
to sff_pio_task.

The only function which is exposed to the core layer is
ata_sff_flush_pio_task() which is renamed from ata_port_flush_task()
and now also takes care of resetting hsm_task_state to HSM_ST_IDLE,
which is possible as it's now specific to PIO HSM.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

ap->[last_]ctl are specific to SFF controllers.  Put them inside
CONFIG_ATA_SFF and move initialization into ata_sff_port_init().
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

->drain_fifo() is SFF specific.  Rename and relocate it.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

In preparation of proper SFF/BMDMA separation, introduce
ata_sff_init/exit() and ata_sff_port_init().  These functions
currently don't do anything.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

When BMDMA initialization failed or BMDMA was not available for
whatever reason, bmdma_addr was left at zero and used as an indication
that BMDMA shouldn't be used.  This leads to the following problems.

p1. For BMDMA drivers which don't use traditional BMDMA register,
    ata_bmdma_mode_filter() incorrectly inhibits DMA modes.  Those
    drivers either have to inherit from ata_sff_port_ops or clear
    ->mode_filter explicitly.

p2. non-BMDMA drivers call into BMDMA PRD table allocation.  It
    doesn't actually allocate PRD table if bmdma_addr is not
    initialized but is still confusing.

p3. For BMDMA drivers which don't use traditional BMDMA register, some
    methods might not be invoked as expected (e.g. bmdma_stop from
    ata_sff_post_internal_cmd()).

p4. SFF drivers w/ custom DMA interface implement noop BMDMA ops
    worrying libata core might call into one of them.

These problems are caused by the muddy line between SFF and BMDMA and
the assumption that all BMDMA controllers initialize bmdma_addr.

This patch fixes p1 and p2 by removing the bmdma_addr assumption and
moving prd allocation to BMDMA port start.  Later patches will fix the
remaining issues.

This patch improves BMDMA initialization such that

* When BMDMA register initialization fails, falls back to PIO instead
  of failing.  ata_pci_bmdma_init() never fails now.

* When ata_pci_bmdma_init() falls back to PIO, it clears
  ap->mwdma_mask and udma_mask instead of depending on
  ata_bmdma_mode_filter().  This makes ata_bmdma_mode_filter()
  unnecessary thus resolving p1.

* ata_port_start() which actually is BMDMA specific is moved to
  ata_bmdma_port_start().  ata_port_start() and ata_sff_port_start()
  are killed.

* ata_sff_port_start32() is moved and renamed to
  ata_bmdma_port_start32().

Drivers which no longer call into PRD table allocation are...

  pdc_adma, sata_inic162x, sata_qstor, sata_sx4, pata_cmd640 and all
  drivers which inherit from ata_sff_port_ops.

pata_icside sets ->port_start to ATA_OP_NULL as it doesn't need PRD
but is a BMDMA controller and doesn't have custom port_start like
other such controllers.

Note that with the previous patch which makes all and only BMDMA
drivers inherit from ata_bmdma_port_ops, this change doesn't break
drivers which need PRD table.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

1. pata_cmd640 is PIO only.  Inherit from sff.

2. pata_macio is BMDMA.  Inherit from bmdma and drop explicit
   bmdma_mode_filter() setting.

3. In sata_mv, unlike mv5, mv6 is BMDMA.  Inherit from bmdma and
   don't clear ->post_internal_cmd().

4. bf54x and icside are quasi-BMDMA controllers which don't use the
   standard BMDMA registers so they don't initialize bmdma_addr and
   inherit from sff to avoid the default mode_filter which disables
   DMA modes if bmdma_addr is not initialized.

For 2 and 3, this patch makes the drivers explicitly specify
->mode_filter to ATA_OP_NULL while inheriting from ata_bmdma_port_ops.
These will be removed by the next patch.

This patch makes all and only BMDMA drivers inherit from
ata_bmdma_port_ops to ease further SFF/BMDMA separation.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

Reorder functions such that SFF and BMDMA functions are grouped.
While at it, s/BMDMA/SFF in a few comments where it actually meant
SFF.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

sata_inic162x doesn't use PRD anymore.  No need to initialize it.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

ATA_FLAG_DISABLED is only used by drivers which don't use
->error_handler framework and is largely broken.  Its only meaningful
function is to make irq handlers skip processing if the flag is set,
which is largely useless and even harmful as it makes those ports more
likely to cause IRQ storms.

Kill ATA_FLAG_DISABLED and makes the callers disable attached devices
instead.  ata_port_probe() and ata_port_disable() which manipulate the
flag are also killed.

This simplifies condition check in IRQ handlers.  While updating IRQ
handlers, remove ap NULL check as libata guarantees consecutive port
allocation (unoccupied ports are initialized with dummies) and
long-obsolete ATA_QCFLAG_ACTIVE check (checked by ata_qc_from_tag()).
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

ata_irq_on() was renamed to ata_sff_irq_on() and exported a while ago
but prototype for the original function lingered in
drivers/ata/libata.h.  Kill it.  Also, ata_dev_select() is only used
inside drivers/ata/libata-sff.c.  Make it static.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

* Clearing IRQ from ata_sff_error_handler() is necessary only when the
  port is gonna be thawed before performing EH actions and some
  controllers don't like being accessed after certain failure modes
  until they're reset.  Clear IRQ iff the port is being thawed.

* When the controller succesfully indicated bus error, the point of
  thawing doesn't matter.  Move thawing inside bmdma part of EH.  This
  is a bit ugly but will ease code reorganization later.

* Remove the unneeded ata_sff_sync().
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

sata_mv initializes unused ioports fields including bmdma_addr to
NULL.  As later changes will conditionalize BMDMA, this makes sata_mv
unnecessarily dependent on BMDMA.  Remove the unnecessary
initialization.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

sata_inic162x no longer uses SFF interface.  Move it out of
CONFIG_ATA_SFF.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

pata_sch is standard SFF.  No reason to open code init.  Use
ata_pci_sff_init_one() instead.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

Use our own mmio area to avoid PCI posting. This avoids the rather slow
paranoid implementation in the default handler.
Signed-off-by: Alan Cox <alan@linux.intel.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

We don't need to stall and wait after loading the task file and before
issuing a command, so don't do it. This shows up on profiles and is not
needed.
Signed-off-by: Alan Cox <alan@linux.intel.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

sata_nv was incorrectly using ata_host_activate() instead of
ata_pci_sff_activate_host() leading to IRQ assignment failure in
legacy mode.  Fix it.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Robert Hancock <hancockr@shaw.ca>
Cc: stable@kernel.org
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

page_mapping() check this via VM_BUG_ON(PageSlab(page)) so we bug here
with the according debuging turned on.

Future TODO: replace this with a flush_dcache_page_for_pio() API
Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>
Cc: stable@kernel.org

cmd640_hardware_init() reads CFR but doesn't use the value read...
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

Now, with the introduction of the sff_set_devctl() method, we can
use it in sff_irq_on() method too -- that way its implementations
in 'pata_bf54x' and 'pata_scc' become virtually identical to
ata_sff_irq_on().  The sff_irq_on() method now becomes quite
superfluous, and the only reason not to remove it completely is
the existence of the 'pata_octeon_cf' driver which implements it
as an empty function. Just make the method optional then, with
ata_sff_irq_on() becoming generic taskfile-bound function, still
global for the 'pata_bf54x' driver to be able to call it from its
thaw() and postreset() methods.

While at it, make the sff_irq_on() method and ata_sff_irq_on() return
'void' as the result is always ignored anyway.
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

The set of libata's taskfile access methods is clearly incomplete as
it lacks a method to write to the device control register -- which
forces drivers like 'pata_bf54x' and 'pata_scc' to implement more
"high level" (and more weighty) methods like freeze() and postreset().

So, introduce the optional sff_set_devctl() method which the drivers
only have to implement if the standard iowrite8() can't be used (just
like the existing sff_check_altstatus() method) and make use of it
in the freeze() and postreset() method implementations (I could also
have used it in softreset() method but it also reads other taskfile
registers without using tf_read() making that quite pointless);
this makes freeze() method implementations in the 'pata_bf54x' and
'pata_scc' methods virtually identical to ata_sff_freeze(), so we
can get rid of them completely.
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

Add "em_buffer" attribute for SATA AHCI hosts to provide a way for
userland to access AHCI EM (enclosure management) buffer directly if the
host supports EM.

AHCI driver should support SGPIO EM messages. However the SATA/AHCI
specs did not define the SGPIO message format filled in EM buffer.
Different HW vendors may have different definitions. The mainly purpose
of this attribute is to solve this issue by allowing HW vendors to
provide userland drivers and tools for their SGPIO initiators.
Signed-off-by: Harry Zhang <harry.zhang@amd.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

Detect enclosure management message type automatically at driver
initialization, instead of using module parameter "ahci_em_messages".
Signed-off-by: Harry Zhang <harry.zhang@amd.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

The device control register exists and its address is set by scc_setup_ports(),
hence the check is useless...
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

... since, of course, it's not used outside this driver.
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

Use __ratelimit() instead of its own private rate limit implementation.
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Cc: Jeff Garzik <jgarzik@pobox.com>
Cc: linux-ide@vger.kernel.org
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

There are some SATA devices which take relatively long to get out of
0xff status after reset.  In libata, this timeout is determined by
ATA_TMOUT_FF_WAIT.  Quantum GoVault is the worst requring about 2s for
reliable detection.  However, because 2s 0xff timeout can introduce
rather long spurious delay during boot, libata has been compromising
at the next longest timeout of 800ms for HHD424020F7SV00 iVDR drive.

Now that parallel scan is in place for common drivers, libata can
afford 2s 0xff timeout.  Use 2s 0xff timeout if parallel scan is
enabled.

Please note that the chance of spurious wait is pretty slim w/ working
SCR access so this will only affect SATA controllers w/o SCR access
which isn't too common these days.

Please read the following thread for more information on the GoVault
drive.

  http://thread.gmane.org/gmane.linux.ide/14545/focus=14663Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Gary Hade <garyhade@us.ibm.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

... since I see no callers of it.
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

In 2009, While running "cache read" performance test of drives behind
SII PMP we encountered a "all 5 drives" timeout on more than 30% of the
machines under test.  This patch reduces the rate by a factor of about 70.
Low enough that we didn't care to further investigate the issue.

Performance impact with any sort of "normal" use was ~2%+ CPU and less
than 1% throughput degradation.  Worst case impact (cached read) was
6% IOPS reduction. This is with NCQ off (q=1) but I believe FIS based
switching enabled in the SATA driver.

The patch disables "Early ACK" in the 3726 port multiplier.
"Early ACK" is issued when device sends a FIS to the host (via PMP)
and the PMP sends an ACK immediately back to the device - well before
the host gets the response. Under worst case IOPs load (cached read
test) and more than 2 PMPs connected to a 4-port SATA controller,
I suspect the time to service all of the PMPs is exceeding the PMPs
ability to keep track of outstanding FIS it owes the Host. Reducing
the number of PMPs to 2 (or 1) reduces the frequency by several orders
of magnitude. Kudos to Gwendal for initial debugging of this issue.
[Any errors in the description are mine, not his.]

Patch is currently in production on Google servers.
Signed-off-by: Grant Grundler <grundler@google.com>
Signed-off-by: Gwendal Grignou <gwendal@google.com>
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

Implicit slab.h inclusion via percpu.h is about to go away.  Make sure
gfp.h or slab.h is included as necessary.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Jeff Garzik <jgarzik@pobox.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

It turns out different generations of MCPs have differing quirks.

* MCP 65-73 : FPDMA AA broken, lies about PMP support, forgets to report NCQ
* MCP 77-79 : FPDMA AA broken, lies about PMP support
* MCP 89    : FPDMA AA broken

Instead of turngin off FPDMA AA on all NVIDIAs, implement
HFLAG_NO_FPDMA_AA, define additional board IDs and apply necessary
quirks.

This fixes bko#15481 and the list of quirks is verified by Peer Chen.

  http://bugzilla.kernel.org/show_bug.cgi?id=15481Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Peer Chen <pchen@nvidia.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

I've prepared a totally simple patch that, if I did it and measured it
correctly, reduces the text size as of the ppc-6xx-size command of
pata-mpc52xx by more than 10%, by reducing the rodata size from 0x4a4
to 0x17e bytes. This is simply done by changing the data types of the
ATA timing constants.

If you are interested at all, and it's worth the trouble, here the
details:

ppc-6xx-size:
     text data bss  dec  hex filename
old: 6532 1068   0 7600 1db0 pata-mpc52xx.o
new: 5718 1068   0 6786 1a82 pata-mpc52xx.o

The (assembler) code itself doesn't really change very much. I double
checked the final results inside mpc52xx-ata-apply-timings() and they
match. The driver is still working fine of course.
Signed-off-by: Roman Fietze <roman.fietze@telemotive.de>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

ahci over time has grown a number of board IDs and it's a bit of mess
right now.  Clean it up such that,

* board_id_* now live in a separate enum board_ids and numbers are
  assigned automatically.

* Board IDs assigned to features are separated from the ones assigned
  to specific implementations and both are ordered alphabetically.

* For NV MCPs, define per-generation alias board_ids and assign
  matching aliases in the pci id table.  This makes mcp_linux, 67-73
  use board_ahci_mcp65 instead of board_ahci_yesncq.  Both are
  identical in content.

* Kill now unused board_ahci_nopmp and board_ahci_yesncq.

This patch doesn't cause any functional change but will make future
changes to board_ids and quirks much less painful.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Peer Chen <pchen@nvidia.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

According to section 10.3.1 of the AHCI spec, PxCMD.ST must not be set
unless there's a device attached. Following this saves us a measurable
quantity of power and does not impair hotplug support. Based on a patch
by Kristen Carlson Accardi.
Signed-off-by: Matthew Garrett <mjg@redhat.com>
Cc: Kristen Carlson Accardi <kristen.c.accardi@intel.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

This can be used for AHCI-compatible interfaces implemented inside
System-On-Chip solutions, or AHCI devices connected via localbus.
Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

This patch should contain no functional changes, just moves code
around.
Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>

Factor out some ahci_em_messages handling code from ahci_init_one().
We would like to reuse it for non-PCI devices.
Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>