On ARMv7 cores, device memory mapped as Normal Non-cacheable, may not guarantee
ordered access causing failures in device drivers that do not use the mandatory
memory barriers. readl & writel versions contain necessary memory barriers for
this.

commit 79f64dbf: "ARM: 6273/1: Add barriers to
the I/O accessors if ARM_DMA_MEM_BUFFERABLE" can be referred for more
information on this.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Now that the dw_dmac DMA driver supports configurable Flow Controller, source
and destination burst or msize, we need to specify which ones to use. Msize or
burst size was previously hardcoded to 1, Flow controller was DMA for both
M2P & P2M transfers.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

This patch sets default Burst length for all transfer to 16. This will
enhance performance when user doesn't have any chan->private data.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Msize or Burst Size is peripheral dependent in case of prep_slave_sg and
cyclic_prep transfers, and in case of memcpy transfers it is platform dependent.
So msize configuration must come from platform data.

Also some peripherals (ex: JPEG), need to be flow controller for dma transfers,
so this information in case of slave_sg & cyclic_prep transfers must come from
platform data.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

src_master & dest_master don't required u32 as they have values limited to u8
only. Also their description is missing from doc style comment. This patch
fixes above mentioned issues.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

In Synopsys designware, channel priority is programmable. This patch adds
support for passing channel priority through platform data. By default Ascending
channel priority will be followed, i.e. channel 0 will get highest priority and
channel 7 will get lowest.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

In SPEAr Platform channels 4-7 have more Fifo depth. So we must get better
channel first. This patch introduces concept of channel allocation order in
dw_dmac. If user doesn't pass anything or 0, than normal (ascending) channel
allocation will follow, else channels will be allocated in descending order.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dwc_desc_get checks all descriptors for DMA_CTRL_ACK before allocating them for
transfers. And descriptors are not marked with DMA_CRTL_ACK after transfer
finishes. Thus descriptor once used is not usable again. This patch marks
descriptors with DMA_CRTL_ACK after dma xfer finishes
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Every descriptor can transfer a maximum count of 4095 (12 bits, in control reg),
So we must have DWC_MAX_COUNT as 4095 instead of 2048.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Lock must be taken before calling dwc_scan_descriptors, as this may
access/modify shared data and queues. dwc_tx_status wasn't taking lock before
calling this routine. This patch add code that takes lock before calling
dwc_scan_descriptors.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dwc_complete_all and other routines was removing all descriptors from dwc->queue
and pushing them to dwc->active_list. Only one was required to be removed. Also
we are calling dwc_dostart, once list is fixed.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

In some cases users of dw_dmac are initialized before dw_dmac, and if they try
to use dw_dmac, they simply fail. So its better we register init() routine
of driver using subsys_initcall() instead of module_init(), so that dma driver
is available at the earliest possible.
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

This driver will now be used in atleast two platforms AVR32 & ARM. And there is
no actual hardware dependency of this driver over AVR32 or ARM. So this
dependency can be removed altogether.

Also dw_dmac driver uses clk framework and must have compilation dependency on
HAVE_CLK
Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

This patch adds dma support for Freescale MXS-based SoC i.MX23/28,
including apbh-dma and apbx-dma.

* apbh-dma and apbx-dma are supported in the driver as two mxs-dma
  instances.

* apbh-dma is different between mx23 and mx28, hardware version
  register is used to differentiate.

* mxs-dma supports pio function besides data transfer.  The driver
  uses dma_data_direction DMA_NONE to identify the pio mode, and
  steals sgl and sg_len to get pio words and numbers from clients.

* mxs dmaengine has some very specific features, like sense function
  and the special NAND support (nand_lock, nand_wait4ready).  These
  are too specific to implemented in generic dmaengine driver.

* The driver refers to imx-sdma and only a single descriptor is
  statically assigned to each channel.
Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

set the number of array correctly.
Signed-off-by: Tomoya MORINAGA <tomoya-linux@dsn.okisemi.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

fix the following kernel error

------------[ cut here ]------------
WARNING: at kernel/softirq.c:159 _local_bh_enable_ip.clone.5+0x35/0x71()
Hardware name: To be filled by O.E.M.
Modules linked in: pch_uart pch_dma fuse mga drm cpufreq_ondemand acpi_cpufreq mperf ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput snd_hda_codec_realtek snd_hda_intel snd_hda_codec matroxfb_base snd_hwdep 8250_pnp snd_seq snd_seq_device matroxfb_DAC1064 snd_pcm joydev 8250 matroxfb_accel snd_timer matroxfb_Ti3026 ppdev pegasus parport_pc snd parport matroxfb_g450 g450_pll serial_core video output matroxfb_misc soundcore snd_page_alloc serio_raw pcspkr ext4 jbd2 crc16 sdhci_pci sdhci mmc_core floppy [last unloaded: scsi_wait_scan]
Pid: 0, comm: swapper Not tainted 2.6.37.upstream_check+ #8
Call Trace:
 [<c0433add>] warn_slowpath_common+0x65/0x7a
 [<c043825b>] ? _local_bh_enable_ip.clone.5+0x35/0x71
 [<c0433b01>] warn_slowpath_null+0xf/0x13
 [<c043825b>] _local_bh_enable_ip.clone.5+0x35/0x71
 [<c043829f>] local_bh_enable_ip+0x8/0xa
 [<c06ec471>] _raw_spin_unlock_bh+0x10/0x12
 [<f82b57dd>] pd_prep_slave_sg+0xba/0x200 [pch_dma]
 [<f82f7b7a>] pch_uart_interrupt+0x44d/0x6aa [pch_uart]
 [<c046fa97>] handle_IRQ_event+0x1d/0x9e
 [<c047146f>] handle_fasteoi_irq+0x90/0xc7
 [<c04713df>] ? handle_fasteoi_irq+0x0/0xc7
 <IRQ>  [<c04045af>] ? do_IRQ+0x3e/0x89
 [<c04035a9>] ? common_interrupt+0x29/0x30
 [<c04400d8>] ? sys_getpriority+0x12d/0x1a2
 [<c058bb2b>] ? arch_local_irq_enable+0x5/0xb
 [<c058c740>] ? acpi_idle_enter_bm+0x22a/0x261
 [<c0648b11>] ? cpuidle_idle_call+0x70/0xa1
 [<c0401f44>] ? cpu_idle+0x49/0x6a
 [<c06d9fc4>] ? rest_init+0x58/0x5a
 [<c089e762>] ? start_kernel+0x2d0/0x2d5
 [<c089e0ce>] ? i386_start_kernel+0xce/0xd5
Signed-off-by: Tomoya MORINAGA <tomoya-linux@dsn.okisemi.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Currently when two or more buffers are queued by the camera driver
and so the double buffering is enabled in the idmac, we lose one
frame comming from CSI since the reporting of arrival of the first
frame is deferred by the DMAIC_7_EOF interrupt handler and reporting
of the arrival of the last frame is not done at all. So when requesting
N frames from the image sensor we actually receive N - 1 frames in
user space.

The reason for this behaviour is that the DMAIC_7_EOF interrupt
handler misleadingly assumes that the CUR_BUF flag is pointing to the
buffer used by the IDMAC. Actually it is not the case since the
CUR_BUF flag will be flipped by the FSU when the FSU is sending the
<TASK>_NEW_FRM_RDY signal when new frame data is delivered by the CSI.
When sending this singal, FSU updates the DMA_CUR_BUF and the
DMA_BUFx_RDY flags: the DMA_CUR_BUF is flipped, the DMA_BUFx_RDY
is cleared, indicating that the frame data is beeing written by
the IDMAC to the pointed buffer. DMA_BUFx_RDY is supposed to be
set to the ready state again by the MCU, when it has handled the
received data. DMAIC_7_CUR_BUF flag won't be flipped here by the
IPU, so waiting for this event in the EOF interrupt handler is wrong.
Actually there is no spurious interrupt as described in the comments,
this is the valid DMAIC_7_EOF interrupt indicating reception of the
frame from CSI.

The patch removes code that waits for flipping of the DMAIC_7_CUR_BUF
flag in the DMAIC_7_EOF interrupt handler. As the comment in the
current code denotes, this waiting doesn't help anyway. As a result
of this removal the reporting of the first arrived frame is not
deferred to the time of arrival of the next frame and the drivers
software flag 'ichan->active_buffer' is in sync with DMAIC_7_CUR_BUF
flag, so the reception of all requested frames works.

This has been verified on the hardware which is triggering the
image sensor by the programmable state machine, allowing to
obtain exact number of frames. On this hardware we do not tolerate
losing frames.

This patch also removes resetting the DMA_BUFx_RDY flags of
all channels in ipu_disable_channel() since transfers on other
DMA channels might be triggered by other running tasks and the
buffers should always be ready for data sending or reception.
Signed-off-by: Anatolij Gustschin <agust@denx.de>
Reviewed-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Tested-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

As per the reference manual, bit "L" should be set while bit "C"
should be cleared for the last buffer descriptor in the non-cyclic
chain, so that sdma can stop trying to find the next BD and end
the transfer.

In case of sdma_prep_slave_sg(), BD_LAST needs to be set and BD_CONT
be cleared for the last BD.
Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

sdma_handle_channel_loop() is the handler of cyclic tx.  One period
success does not really mean the success of the tx.  Instead of
DMA_SUCCESS, DMA_IN_PROGRESS should be the one to tell.
Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

The sdmac->status was designed to reflect the status of the tx,
so simply return it in sdma_tx_status().  Then dma client can call
dma_async_is_tx_complete() to know the status of the tx.
Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

sdma_prep_dma_cyclic() sets sdmac->status to DMA_ERROR in err_out,
and sdma_prep_slave_sg() needs to do the same.  Otherwise,
sdmac->status stays at DMA_IN_PROGRESS, which will make the function
return immediately next time it gets called.
Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

This is a leftover from the time that the driver did not have
sdma_prep_dma_cyclic callback and implemented sound dma as a looped
sg chain.  And it can be removed now.
Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

The capabilities are device specific fields, not channel specific fields.
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

We need channel 0 of the sdma engine for internal purposes. We
accomplished this by calling dma_request_channel() in the probe
function. This does not work when multiple dma engines are
present which is the case when IPU support for i.MX31/35 is
compiled in. So instead of registering channel 0 and reserving
it afterwards simply do not register it in the first place.
With this the dmaengine channel counting does not match sdma
channel counting anymore, so we have to use sdma channel counting
in the driver.
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

The capabilities are device specific fields, not channel specific fields.
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

This is bogus as the dmaengine core will overwrite this field.
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

This patch lets sdma_prep_slave_sg fail if the entries of an
sg list do not start on multiples of the word size or if the
lengths are not multiple of the word size.
Also, catch the previously unhandled DMA_SLAVE_BUSWIDTH_8_BYTES
and DMA_SLAVE_BUSWIDTH_UNDEFINED cases.
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

Support cyclic transfers, which are useful for ALSA drivers.
Acked-by: Per Forlin <per.forlin@stericsson.com>
Acked-by: Jonas Aaberg <jonas.aberg@stericsson.com>
Signed-off-by: Rabin Vincent <rabin.vincent@stericsson.com>
Signed-off-by: Linus Walleij <linus.walleij@stericsson.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

The current implementation of DMA_TERMINATE_ALL leaves ongoing transfers
running.  Fix it.
Acked-by: Per Forlin <per.forlin@stericsson.com>
Acked-by: Jonas Aaberg <jonas.aberg@stericsson.com>
Signed-off-by: Rabin Vincent <rabin.vincent@stericsson.com>
Signed-off-by: Linus Walleij <linus.walleij@stericsson.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

The DMA_SG cap is enabled on the wrong channel, and the pointers are repeatedly
set incorrectly.  Fix it and combine the ops settings to a common function.
Acked-by: Per Forlin <per.forlin@stericsson.com>
Acked-by: Jonas Aaberg <jonas.aberg@stericsson.com>
Signed-off-by: Rabin Vincent <rabin.vincent@stericsson.com>
Signed-off-by: Linus Walleij <linus.walleij@stericsson.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>