"Hello, world" with Mach threads

Questions and Answers about all things *OS (macOS, iOS, tvOS, watchOS)

"Hello, world" with Mach threads

Postby voidpointer » Wed Aug 20, 2014 5:09 pm

Hello everyone,

I would like to use Mach threads in one of my applications. However, I'm having trouble getting a simple "Hello, world" program using Mach threads to work on OS X 10.9.4. The following MWE always crashes with a segmentation fault. Perhaps I am not setting up the stack correctly, but I can't figure out where I'm going wrong. Do you have any ideas?

Thanks for your help!

Code: Select all
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>

#include <unistd.h>
#include <mach/mach_init.h>
#include <mach/mach_types.h>
#include <mach/task.h>
#include <mach/thread_act.h>
#include <mach/thread_policy.h>
#include <mach/i386/thread_status.h>

void check(kern_return_t err)
{
        if (err == KERN_SUCCESS) {
                return;
        }

        switch (err) {
        case KERN_FAILURE:
      fprintf(stderr, "failure");
        case KERN_INVALID_ARGUMENT:
      fprintf(stderr, "invalid argument");
        default:
      fprintf(stderr, "unknown error");
        }
   exit(EXIT_FAILURE);
}

void test()
{
   printf("Hello, world!");
}

int main()
{
   thread_t thread;
        int page_size = getpagesize();
        uint8_t* stack = (uint8_t*)malloc(page_size);
        task_t task = mach_task_self();
        check(thread_create(task, &thread));

        x86_thread_state64_t state;
        mach_msg_type_number_t count = x86_THREAD_STATE64_COUNT;
        check(thread_get_state(thread, x86_THREAD_STATE64,
                (thread_state_t)&state, &count));

        uintptr_t stack_ptr = (uintptr_t)(stack + page_size);
        stack_ptr &= -16;
        stack_ptr -= sizeof(void*);

        state.__rip = (uintptr_t)test;
        state.__rsp = (uintptr_t)stack_ptr;
        state.__rbp = (uintptr_t)stack_ptr;
        check(thread_set_state(thread, x86_THREAD_STATE64,
                (thread_state_t)&state, x86_THREAD_STATE64_COUNT));

        check(thread_resume(thread));
   // Give the thread a chance to run.
        sleep(1);
   printf("Done.");
}
voidpointer
 
Posts: 6
Joined: Wed Aug 20, 2014 4:52 pm

Re: "Hello, world" with Mach threads

Postby morpheus » Sat Aug 23, 2014 4:57 am

My immediate suspect would be that you need to convert Mach threads to Pthreads in order to use std* functions (like printf). You can look at the code injection sample I put up a while ago in the bonus section. Said "promotion" wasn't explained in the book, but surely will be, in the second edition.
morpheus
Site Admin
 
Posts: 532
Joined: Thu Apr 11, 2013 6:24 pm

Re: "Hello, world" with Mach threads

Postby voidpointer » Sat Aug 23, 2014 5:54 pm

Thanks for the reply! I don't think it's the call to printf that's the problem: I removed the call to printf (so that the test function is empty), and allocated the stack, adjusted the stack pointer, and created the thread in the same way that is done in the code injection example. Unfortunately, the segmentation fault still persists. I have included the updated C++11 MWE below. In case you want to compile it, you can do so using clang++ -std=c++11 <file>.

Code: Select all
#include <cstdint>
#include <iostream>
#include <system_error>

#include <unistd.h>
#include <mach/mach.h>
#include <mach/mach_vm.h>
#include <mach/mach_error.h>
#include <mach/mach_init.h>
#include <mach/mach_types.h>

#include <mach/task.h>
#include <mach/thread_act.h>
#include <mach/thread_policy.h>
#include <mach/i386/thread_status.h>

void check(kern_return_t err)
{
        if (err == KERN_SUCCESS) return;
   throw std::runtime_error{::mach_error_string(err)};
}

static void test() {}

int main()
{
        auto thread     = ::thread_t{};
        auto task       = ::mach_task_self();
        auto stack_size = 65536;
        auto stack      = (::vm_address_t)nullptr;
   check(::mach_vm_allocate(task, (::mach_vm_offset_t*)&stack, stack_size,
      VM_FLAGS_ANYWHERE));

        auto state     = ::x86_thread_state64_t{};
        auto count     = ::mach_msg_type_number_t{x86_THREAD_STATE64_COUNT};
        auto stack_ptr = stack + stack_size / 2 - 8;
        state.__rip = (uintptr_t)test;
        state.__rsp = (uintptr_t)stack_ptr;
        state.__rbp = (uintptr_t)stack_ptr;

   check(::thread_create_running(task, x86_THREAD_STATE64,
      (thread_state_t)&state, x86_THREAD_STATE64_COUNT, &thread));
        ::sleep(1);
        std::cout << "Done." << std::endl;
   // TODO free thread reference
}
voidpointer
 
Posts: 6
Joined: Wed Aug 20, 2014 4:52 pm

Re: "Hello, world" with Mach threads

Postby morpheus » Thu Aug 28, 2014 7:16 pm

Ill have a look see. I'm currently traveling with no clang (still good ol' GCC) so I'll need some time to make this compile in plain C and resolve the issue.
morpheus
Site Admin
 
Posts: 532
Joined: Thu Apr 11, 2013 6:24 pm

Re: "Hello, world" with Mach threads

Postby voidpointer » Fri Aug 29, 2014 3:44 am

Included below is the listing converted to C99. The code successfully compiles under GCC and Clang. I have also made a Github Gist here: https://gist.github.com/adityaramesh/7cf7ec54ce9e32041020

Code: Select all
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>

#include <unistd.h>
#include <mach/mach.h>
#include <mach/mach_vm.h>
#include <mach/mach_error.h>
#include <mach/mach_init.h>
#include <mach/mach_types.h>

#include <mach/task.h>
#include <mach/thread_act.h>
#include <mach/thread_policy.h>
#include <mach/i386/thread_status.h>

void check(kern_return_t err)
{
   if (err == KERN_SUCCESS) return;
   fprintf(stderr, mach_error_string(err));
   exit(EXIT_FAILURE);
}

void test() {}

int main()
{
   thread_t thread;
   task_t task        = mach_task_self();
   int stack_size     = 65536;
   vm_address_t stack = 0;
   check(mach_vm_allocate(task, (mach_vm_offset_t*)&stack, stack_size,
      VM_FLAGS_ANYWHERE));
   
   x86_thread_state64_t state;
   mach_msg_type_number_t count = x86_THREAD_STATE64_COUNT;
   vm_address_t stack_ptr = stack + stack_size / 2 - 8;

   state.__rip = (uintptr_t)test;
   state.__rsp = (uintptr_t)stack_ptr;
   state.__rbp = (uintptr_t)stack_ptr;
   check(thread_create_running(task, x86_THREAD_STATE64,
      (thread_state_t)&state, x86_THREAD_STATE64_COUNT, &thread));

   // Give the thread a chance to run.
   sleep(1);
   printf("Done.");
}
voidpointer
 
Posts: 6
Joined: Wed Aug 20, 2014 4:52 pm

Re: "Hello, world" with Mach threads

Postby morpheus » Sun Aug 31, 2014 11:49 pm

Your segfault is for two different reasons:


- Without anything in test(), the function actually does execute, then returns - i.e. pops the RA - but you didn't set anything on the stack, so you get a POP of a NULL value, hence a segfault.
- With something (e.g. printf()) in test, you fail, as I mentioned earlier, because of pthread_getspecific, since you did not convert the mach thread to a pthread thread.
Code: Select all

* thread #2: tid = 0x21db2a, 0x00007fff8d894128 libsystem_c.dylib`pthread_getspecific, stop reason = EXC_BAD_ACCESS (code=1, address=0x50)
    frame #0: 0x00007fff8d894128 libsystem_c.dylib`pthread_getspecific
libsystem_c.dylib`pthread_getspecific:
-> 0x7fff8d894128:  movq   %gs:(,%rdi,8), %rax
   0x7fff8d894131:  ret   
   0x7fff8d894132:  nop   
   0x7fff8d894133:  nop   
(lldb) bt
* thread #2: tid = 0x21db2a, 0x00007fff8d894128 libsystem_c.dylib`pthread_getspecific, stop reason = EXC_BAD_ACCESS (code=1, address=0x50)
  * frame #0: 0x00007fff8d894128 libsystem_c.dylib`pthread_getspecific
    frame #1: 0x00007fff8d91382f libsystem_c.dylib`printf + 171
    frame #2: 0x0000000100000cf6 t`test + 22
(lldb)
morpheus
Site Admin
 
Posts: 532
Joined: Thu Apr 11, 2013 6:24 pm

Re: "Hello, world" with Mach threads

Postby voidpointer » Mon Sep 01, 2014 4:16 am

Thanks so much for taking the time to look into this, and for introducing me to LLDB! It has been immensely helpful.

I am now able to get the "Hello, world!" to show up using the raw write() system call. Since I don't want to return to anything from test(), I would like to tell the kernel to terminate the current thread. The following code results in a crash after printing "Hello, world!", despite the fact that mach_thread_self() is returning the correct thread ID (which I confirmed).

Code: Select all
void test()
{
   write(1, "Hello, world!\n", 14);
   check(thread_terminate(mach_thread_self()));
        // Just to illustrate that this crash is not because of popping an invalid RA.
   for (;;);
}


Code: Select all
(lldb) r
Process 1505 launched: './a.out' (x86_64)
Hello, world!
Process 1505 stopped
* thread #2: tid = 0x1b1f6, 0x00007fff835ccc72 libsystem_kernel.dylib`mig_get_reply_port + 4, stop reason = EXC_BAD_ACCESS (code=1, address=0x10)
    frame #0: 0x00007fff835ccc72 libsystem_kernel.dylib`mig_get_reply_port + 4
libsystem_kernel.dylib`mig_get_reply_port + 4:
-> 0x7fff835ccc72:  movq   %gs:0x10, %rax
   0x7fff835ccc7b:  testl  %eax, %eax
   0x7fff835ccc7d:  jne    0x7fff835ccc8f            ; mig_get_reply_port + 33
   0x7fff835ccc7f:  callq  0x7fff835cc9e0            ; mach_reply_port
(lldb) bt
* thread #2: tid = 0x1b1f6, 0x00007fff835ccc72 libsystem_kernel.dylib`mig_get_reply_port + 4, stop reason = EXC_BAD_ACCESS (code=1, address=0x10)
  * frame #0: 0x00007fff835ccc72 libsystem_kernel.dylib`mig_get_reply_port + 4
    frame #1: 0x00007fff835c8943 libsystem_kernel.dylib`thread_terminate + 24
    frame #2: 0x0000000100000dd8 a.out`test() + 40 at test.cpp:38


You mentioned in your book that "the implementation of get_ active_thread() wraps CPU_DATA_GET(cpu_active_thread,thread_t), which is inline assembly (relying on the GS register)". I tried to search online to get any hints as to what the cause of the problem is, but was unable to solve the issue. Do you know how I would go about correctly terminating the thread at the end of test()?
voidpointer
 
Posts: 6
Joined: Wed Aug 20, 2014 4:52 pm

Re: "Hello, world" with Mach threads

Postby Emmanuel » Thu Sep 11, 2014 9:04 pm

Like Administrator said, if you don't use pthread_set_self() at the beginning of your mach thread, you will eventually crash.
The issue is that a lot of functions will make a call to (g/s)et_thread_specifics() and that will crash because GS is not properly set (will be 0). Therefore, you get a segmentation fault or something like that.

You certainly can use native functions but you will need to figure out which function make references to GS and which don't.
Emmanuel
 
Posts: 7
Joined: Thu Jul 18, 2013 4:41 pm

Re: "Hello, world" with Mach threads

Postby voidpointer » Mon Nov 17, 2014 4:57 am

I apologize for the very late reply -- I only recently got the time to properly look into what you suggested. I have two questions regarding pthread_set_self that I could use some help answering. Based on information I gathered about x64 system call conventions for OS X and the value of the SYSCALL_MDEP_CONSTRUCT macro, I think I am supposed to call pthread_set_self (which is thread_fast_set_cthread_self64 for x64) as follows:

Code: Select all
asm volatile(
    "movq $50331651, %rax\n\t"
    "movq <thread ID>, %rdi\n\t"
    "syscall"
);


I obtained the constant 50331651 by expanding SYSCALL_MDEP_CONSTRUCT(3), since thread_fast_set_cthread_self64 occupies the third slot in the machine-dependent system call table on x64. The value moved into RDI should be replaced by the value to which we wish to set the thread ID. Is this right?

Also, to which value do I set the thread ID? I'm pretty sure it's not the value returned by thread_create, so what should it be?

Thanks again for your help.
voidpointer
 
Posts: 6
Joined: Wed Aug 20, 2014 4:52 pm

Re: "Hello, world" with Mach threads

Postby voidpointer » Fri Nov 21, 2014 9:19 pm

I was finally able to find a solution to my problem*, but the solution is not very favorable. I am including this information here in case it is useful to others who are facing similar problems. While reading some system header files, I came across the function pthread_create_suspended_np, which sets up a pthread with the underlying Mach thread in a suspended state. This does the nontrivial task of filling in the fields of the _opaque_pthread_t structure with the right values and calling pthread_set_self with a pointer to this structure. (For more information about pthread_set_self, see The Mac Hacker’s Handbook, pages 301–305.) After using this function, I assigned each thread a distinct affinity tag, and then ran all of the threads using mach_thread_resume. As it turns out, it is still not possible to force each thread to run on its own processor. It looks like this is the best we can do for now =/

* I put up a code listing that does everything in the above paragraph here: https://gist.github.com/adityaramesh/8a ... 4a0db411ac. Each thread prints out the x2APIC ID of the processor on which it is running. In case you would like to compile the listing, you will need the small library available here: https://github.com/adityaramesh/ctop.
voidpointer
 
Posts: 6
Joined: Wed Aug 20, 2014 4:52 pm


Return to Questions and Answers

Who is online

Users browsing this forum: No registered users and 1 guest