If ioreamp_prot() fails in ioremap_page_range() due to kernel memory
exhaustion, we previously would leak a struct vm_struct.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

This change creates the framework for vDSO calls, makes the existing
rt_sigreturn() mechanism use it, and adds a fast gettimeofday().
Now that we need to expose the vDSO address to userspace, we add
AT_SYSINFO_EHDR to the set of aux entries provided to userspace.
(You can disable any extra vDSO support by booting with vdso=0,
but the rt_sigreturn vDSO page will still be provided.)

Note that glibc has supported the tile vDSO since release 2.17.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

The tile code notifies the simulator of new ET_EXEC objects starting
to execute so that tracing code can properly annotate the objects.
However, we didn't support ET_DYN executables like ld.so, so we
didn't properly load symbols, etc.  This change enables that support;
we use a variant of the SIM_CONTROL_DLOPEN simulator notification
that newer simulators will recognize and use to set the base address
for the next SIM_CONTROL_OS_EXEC notification.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

First, don't re-enable interrupts blindly in the Linux trap handler.
We already handle page faults this way; synchronous interrupts like
ILL_TRANS will fire even when interrupts are disabled, and we don't
want to re-enable interrupts in that case.

For ILL_TRANS, we now pass the ILL_VA_PC reason into the trap handler
so we can report it properly; this is the address that caused the
illegal translation trap.  We print the address as part of the
pr_alert() message now if it's coming from the kernel.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

It's much easier to read register dumps if you read vertically
rather than horizontally, since the register numbers line up
and lead the eye down more than to the right.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

First, fix a bug in asm/unaligned.h; we need to just use the asm-generic
unaligned.h so we properly choose endian-correct flavors.

Second, keep the hv/hypervisor.h ABI fully "native" in the sense that
we don't have __BIG_ENDIAN__ ifdefs there.  Instead, we use macros in
the head_NN.S assembly code to properly extract two 32-bit structure
members from a 64-bit register holding the structure.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

This change adds support for CONFIG_PREEMPT (full kernel preemption).
In addition to the core support, this change includes a number
of places where we fix up uses of smp_processor_id() and per-cpu
variables.  I also eliminate the PAGE_HOME_HERE and PAGE_HOME_UNKNOWN
values for page homing, as it turns out they weren't being used.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Since it's a no-op on tile anyway, there's no reason to be calling
it in tile-specific code.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

First, in huge_pte_offset(), we were erroneously checking
pgd_present(), which is always true, rather than pud_present(),
which is the thing that tells us if there is a top-level (L0) PTE.
Fixing this means we properly look up huge page entries only when
the Present bit is actually set in the PTE.

Second, use the standard pte_alloc_map() instead of the hand-rolled
pte_alloc_hugetlb() routine that basically was written to avoid
worrying about CONFIG_HIGHPTE.  However, we no longer plan to support
HIGHPTE, so a separate routine was just unnecessary code duplication.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Also, alphabetize the existing entries for tile.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

This change adds support for avoiding recursive backtracer crashes;
we haven't seen this in practice other than when things are seriously
corrupt, but it may help avoid losing the root cause of a crash.

Also, don't abort kernel backtracers for invalid userspace PC's.
If we do, we lose the ability to backtrace through a userspace
call to a bad address above PAGE_OFFSET, even though that it can
be perfectly reasonable to continue the backtrace in such a case.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

This change enables unaligned userspace memory access via a kernel
fast path on tilegx.  The kernel tracks user PC/instruction pairs
per-thread using a direct-mapped cache in userspace.  The cache
maps those PC/instruction pairs to JIT'ed instruction sequences that
load or store using byte-wide load store intructions and then
synthesize 2-, 4- or 8-byte load or store results.  Once an
instruction has been seen to generate an unaligned access once,
subsequent hits on that instruction typically require overhead
of only around 50 cycles if cache and TLB is hot.

We support the prctl() PR_GET_UNALIGN / PR_SET_UNALIGN sys call to
enable or disable unaligned fixups on a per-process basis.

To do this we pull some of the tilepro unaligned support out of the
single_step.c file; tilepro uses instruction disassembly for both
single-step and unaligned access support.  Since tilegx actually has
hardware singlestep support, though, it's cleaner to keep the tilegx
unaligned access code in a separate file.  While we're at it,
properly rename the tilepro-specific types, etc., to have tilepro
suffixes instead of generic tile suffixes.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Pointed out by checkpatch.  A few of the DEFINE() lines were
properly written without backslash continuation; fix the rest.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

As specified, the test wasn't correct, and in any case it should
be a BUILD_BUG_ON.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

This change adds support for the "memmap" boot parameter similar
to what x86 provides.  The tile version supports "memmap=1G$5G",
for example, as a way to reserve a 1 GB range starting at PA 5GB.
The memory is reserved via bootmem during startup, and we create a
suitable "struct resource" marked as "Reserved" so you can see the
range reported by /proc/iomem.  Up to 64 such regions can currently
be reserved on the boot command line.

We do not support the x86 options "memmap=nn@ss" (force some memory
to be available at the given address) since it's pointless to try to
have Linux use memory the Tilera hypervisor hasn't given it.  We do
not support "memmap=nn#ss" to add an ACPI range for later processing,
since we don't support ACPI.  We do not support "memmap=exactmap"
since we don't support reading the e820 information from the BIOS
like x86 does.  I did add support for "memmap=nn" (and the synonym
"mem=nn") which cap the highest PA value at "nn"; these are both
just a synonym for the existing tile boot option "maxmem".
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

This change improves and cleans up the tile console.

- We enable HVC_IRQ support on tilegx, with the addition of a new
  Tilera hypervisor API for tilegx to allow a console IPI.  If IPI
  support is not available we fall back to the previous polling mode.

- We simplify the earlyprintk code to use CON_BOOT and eliminate some
  of the other supporting earlyprintk code.

- A new tile_console_write() primitive is used to send output to
  the console and is factored out of the hvc_tile driver.
  This lets us support a "sim_console" boot argument to allow using
  simulator hooks to send output to the "console" as a slightly
  faster alternative to emulating the hardware more directly.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

We can take advantage of the fact that bit 29 is hard-wired
to zero in register TRIO_TILE_PIO_REGION_SETUP_CFG_ADDR.
This is handy since at the moment we only allocate one 4GB
region for vmalloc, and with this change we can allocate
four or more TRIO MACs without using up all the vmalloc space.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

On Tilera Gx72 systems, the logic for figuring out whether
a given port is root complex is slightly different.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

The code incorrectly masked with PAGE_OFFSET instead of
PAGE_SIZE-1.  This only matters when trying to do a
non page-aligned DMA; it was noticed during code inspection.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

The standard kernel function dma_get_required_mask() uses the
highest DRAM address to determine if 32-bit or 64-bit DMA addressing
is needed.  This only works on architectures that have direct mapping
between the PA and the PCI address space, i.e. those that don't have
I/O TLBs or have I/O TLB but choose to use direct mapping.  Neither
of these are true for tilegx.  Whether to use 64-bit DMA should depend
on the PCI device's capability only, not on the amount of DRAM
installeds, so we now advertise a 64-bit DMA mask unconditionally.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Using the low-level hv_dev_pread() API makes assumptions about the
layout of datastructures in the Tilera hypervisor API; it's better to
use the gxio_XXX accessor and the pcie_trio_ports_property struct.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

This change is purely stylistic but improves the readability
of the tile PCI RC driver.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

The .mem_resources[] field in the pci_controller struct
is now obsoleted by the .mem_space and .io_space fields.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

The TRIO shim initialization is shared with other kernel drivers
such as the endpoint and StreamIO drivers, so reorganize the
initialization flow to ensure that the root complex driver properly
initializes TRIO state regardless of what kind of TRIO driver will
end up using the shim.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

To enable this functionality, configure CONFIG_TILE_PCI_IO.  Without
this flag, the kernel still assigns I/O address ranges to the
devices, but no TRIO resource and mapping support is provided.

We assign disjoint I/O address ranges to separate PCIe domains.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Besides using pr_info() to print the linkdown status for a plug-in
slot, add extra indication that this is expected if the slot is empty.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

To support PCIe devices with higher number of MSI-X interrupt vectors,
e.g. 16 for the LSI RAID card, enhance the Gx RC stack to provide more
MSI-X vectors by using the TRIO Scatter Queues, which provide 8 more
vectors in addition to ~10 from the Map Mem regions.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

The LSI MEGARAID SAS HBA suffers from the problem where it can do
64-bit DMA to streaming buffers but not to consistent buffers.
In other words, 64-bit DMA is used for disk data transfers and 32-bit
DMA must be used for control message transfers. According to LSI,
the firmware is not fully functional yet. This change implements a
kind of hybrid dma_ops to support this.

Note that on most other platforms, the 64-bit DMA addressing space is the
same as the 32-bit DMA space and they overlap the physical memory space.
No special arrangement is needed to support this kind of mixed DMA
capability.  On TILE-Gx, the 64-bit DMA space is completely separate
from the 32-bit DMA space.  Due to the use of the IOMMU, the 64-bit DMA
space doesn't overlap the physical memory space.  On the other hand,
the 32-bit DMA space overlaps the physical memory space under 4GB.
The separate address spaces make it necessary to have separate dma_ops.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

If we are rebooting (e.g. via kexec) then the PCI RC link may
already be up.  In that case, we don't want to do the software
fixup to force the link up, since that can degrade it to Gen1.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Allow longer delays if requested, and print the info messages
as we are performing the delay, not when parsing the arguments.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Fix a bug in the tilepro PCI resource allocation code that could make
the bootmem allocator unhappy if 4GB is installed on mshim 0.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

- remove unneeded <linux/bootmem.h> include in pci.c
- eliminate unused pci_controller.first_busno field
- prefer msleep to mdelay
- remove stale comment about pci_scan_bus_parented()
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

Using strlen as a model, add length checking to create strnlen.
Signed-off-by: Ken Steele <ken@tilera.com>
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

This change cleans up the string code in a number of ways:

- For memcpy(), fix bug in prefetch and increase distance to 3 lines;
  optimize for unaligned data; do all loads before wh64 to make memcpy
  safe for forward-overlapping calls; etc.  Performance is improved.

- Use new copy_byte() function on tilegx to spread a single byte value
  out into a full word using the shufflebytes instruction.

- Clean up header include ordering to be more canonical, and remove
  spurious #undefs of function names.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>

The "inv" (invalidate) instruction is generally less safe than "finv"
(flush and invalidate), as it will drop dirty data from the cache.
It turns out we have almost no need for "inv" (other than for the
older 32-bit architecture in some limited cases), so convert to
"finv" where possible and delete the extra "inv" infrastructure.
Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>